The allegations of contamination (using Tristan and Levent's work) aren't very well evidenced, but this behavior by OpenAI (from the authors' statement) makes them seem like the bad guys:
> I said that if OpenAI released its result in the way proposed I would go
public with what happened. The reply was, “Why would you ruin your career?”
I replied that I am an academic, and asked why he thought going public would
ruin my career. The reply was, “If you don’t want me to be nice, then I don’t
have to be nice.”
Threatening a research mathematician and dangling and $1M payday to dissociate from his research collaborators and to adopt OpenAI's narrative is bad stuff.
(To help people keep track: that's OpenAI (allegedly) threatening Tristan Buckmaster (NYU) to remove Levent Alpöge as a co-author. Alpöge is a well-known[0] Anthropic mathematician).
Interesting that they quote the mathematician directly: “there is nothing you can do, I simply do not trust you”
but then they proceed to NOT quote themselves themselves verbatim: "I deeply apologize for this extremely poor choice of words, it is the opposite of what I was trying to convey."
> "I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."
OpenAI (i.e. this OP):
> "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ."
Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data?
This is one of the major problems with these enormous closed models, and even most open-weights models, which don't disclose their training process or training data. You can never be sure what went into its training. Did it come up with an idea originally, or is it just plagiarising its training data? Are there malicious inputs being used to train in particular behaviors when given certain trigger phrases? What are the characteristics of the RLHF data and what kind of biases are those embedding in the models?
With proprietary closed models, or even open weights models that don't have open training datasets, you just can't answer these questions.
A careful reading of "we cannot rule out that de-identified data derived from their usage of our products helped improve our models" could be saying that yes they trained on it but they don't know if that training data resulted in an "improvement" to the model. That is, they can't rule out that the only reason the model found this solution was because it had been trained on this approach.
The term ruled out is very open ended and gives them significant flexibility of meaning. They may have the information to determine exactly what happened, but they haven't looked so they can't "rule it out".
To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effect on model behavior.
As a parallel example, can we prove the phase of the moon had no impact on the NS solution? No, not without a bunch experiments run at different phases of the moon.
There's no reason to believe that anything they did in ChatGPT led to our solution; it's just impossible for us to truly prove it. And knowing most of the recipes we use, there's really no reason to think such contamination is possible. I've asked the team to make a clearer, less-lawyerly statement here - let's see what happens.
> Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data?
Probably? I have a few hundred TB of training data for various small scale models and I can attest that I have _no idea_ what's in them. As in, literally zero. Half is scraped from GitHub and other hosting sites, other than that, I couldn't tell you anything else.
At OpenAI's scale their entire pipeline is likely 100% automated.
But that doesn't preclude being able to index and track what the sources of data are. For your data sets, I would hope you are including source information for where the data came frome. And at OpenAI's scale, I would presume they are doing some amount of rolling hashing or similar to weed out duplication, training on too much duplicate data can cause problems.
AllenAI have at least attempted to add some amount of traceability to their models with OLMoTrace (https://arxiv.org/abs/2504.07096), by letting you find n-gram matches from the outputs in their training data. It's not the most useful, there's a reason that LLMs use full fledged attention mechanisms and not just n-grams, a lot of times the n-gram matches it finds aren't all that related to the given output, it might be better to supplement this index with a vector search or other ways of keeping track of what training data would have most influenced particular parts of the output.
But anyhow, this is something that is an important question, and the big labs should be working on to make their products more trustworthy. Instead, they are hiding information about how they train, hiding their reasoning traces, and just producing output with no information on what might have influenced the training.
Aye, but do they train on user data in these circumstances or not? If they do, then almost certainly the model was influenced by the input of the allegedly plagiarised material.
OAI could check whether those accounts enabled training data. If "yes", OAI could trace whether that data was used in any related training process. If either of those answers comes out to be "no", then that's sufficient to conclude training data independence.
We wouldn't need a full ablated re-training and solution attempt, contra tedsanders in a sibling comment.
I feel like it's far more likely that ordinary corporate espionage or leak led to this rather than OpenAI sifting through piles of user data to find this approach. Buckmaster's collaborator works at Anthropic, and could have been targeted. That would also explain why they aren't forthcoming with the source of the prompt.
I thought one of the issues was that they wanted to remove credit from Levant, the aforementioned Anthropic collaborator? Which doesn't make sense to me if he was leaking information, or defecting to OpenAI, but I might be misunderstanding your point.
I believe jrflo was saying that OpenAI watches the chats of everyone from Anthropic because watching what Anthropic employees type into their personal ChatGPT accounts is a critical source of intelligence on is happening inside of Anthropic.
I would be surprised if OpenAI isn't doing that. OpenAI will take any advantage they can get. If an employee at their primary adversary is typing useful intelligence into OpenAIs website, a website that does not promise privacy from OpenAI, the only reason they wouldn't weaponize that information against Anthropic is ethics or fair play.
I don't think he was defecting or leaking directly, just that it's entirely possible that this information got to OpenAI as a rumor rather than them directly spying on mathematicians chat logs.
The famous Oracle of Delphi in Ancient Greece was said to be the center of the universe in its time. Kings, generals, and officials from poleis across and from without Greece would seek the Oracle’s counsel on important decisions.
Stories of Apollo’s favor and hallucinogenic gases abound, but I think the late Yale professor of Ancient Greek history, Donald Kagan, explained it best:
“Now, you can bet when these folks came and consulted the priests and said, ‘could you please put us down on the list, we want to consult the oracle’, the priests said ‘sure, have a beer, let's talk about your hometown, what's going on out there’. What I'm suggesting to you is that this was the best information gathering and storing device that existed in the Mediterranean world. These people knew more than anybody else about these things, and so consulting that oracle was a very rational act indeed.”
Given how OpenAI models break free of their safeguards and hack others to game their scores..
.. can they really know it didn't do the same inadvertently when they prompted things like "someone is close to solving this problem using our tools, try to beat them", and it then decides to hack and peek at their own chats..?
Yes, wild speculation. But warranted, I feel, given OpenAIs behavior.
> I should also emphasize that this is not an institutional effort. It is a strictly personal collaboration between the two of us, and there is no formal agreement behind it. I pay for the tools my group uses out of my own research funds, including footing a large bill to OpenAI.
The non-Anthropic employee, Tristan Buckmaster, is the one paying for OpenAI models and presumably the one who chose to use them. The Anthropic employee, Levent Alpöge, was collaborating in his personal capacity, and obviously it wouldn't make sense for him to cut off their work together just because his employer's competitor's tool was used.
I suspect this controversy will blow the case for ZDR wide open. Whatever the facts (possibly unknowable), it's going to become a very public lesson that data sovereignty was never about "having nothing to hide".
If this is what they do to academic pure mathematicians, where the stakes are so low (financially)—just imagine the sort of front-running that could be happening in other places.
They do mention that in the "Concurrent Work" section.
Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge, an Anthropic employee, and Tristan Buckmaster, a math professor at NYU. After the completion of our full project and Lean verification (on September 6th), believing from the rumor they also had a solution of Navier–Stokes, we reached out to them to offer a concurrent release of our result and to recognize their priority in a joint announcement. At that point we found out that they had a resolution of the forced Euler problem. In these discussions we offered them visibility into all of the prompts we used and later to see the proof. We recognize the priority of their work on forced Euler and congratulate them on their remarkable mathematical achievement.
"I was shown a prompt and told the internal research model had simply been
given the problem statement. Levent had been told by Sebastien “very little
human input” had been used. This turned out not to be true. Over the course
of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute
had been used."
One of the interesting threads here that is certainly relevant to the OpenAI writeup is the human role in the process. Buckmaster clearly points out that (exceptional!) mathematicians at OpenAI were certainly involved in correcting and guiding the process - and that their path/strategy was no doubt influenced by Alpoge & Buckmaster's work. It is always in OpenAI's interest to de-emphasize the role of people in the process, as is clearly the case here. Indeed, given sufficient compute and resources, I suspect Buckmaster could have also extended their approach to N-S.
It reminds me of the Cognitive Dark Forest hypotheses recently shared here:
> “You are creating your cool streaming platform in your bedroom. Nobody is stopping you, but if you succeed, if you get the signal out, if you are being noticed, the large platform with loads of cash can incorporate your specific innovations simply by throwing compute and capital at the problem. They can generate a variation of your innovation every few days, eventually they will be able to absorb your uniqueness. It’s just cash, and they have more of it than you.
So the safest bet again is to stay silent, or at least under the radar. Best bet is to not disrupt - succeed at all … ?”
To my understanding, those mathematicians proved a subset of problems, not the Navier-Stokes problem itself. OpenAI used that subproblem in its proof of NS it seems.
The drama comes from where OpenAI got the idea to use that route to tackle NS, since the authors maintain that no one could have plucked it out of thin air like the OpenAI research claim to have done.
“ There does not seem to be anything in principle preventing the methods from extending all the way to Navier-Stokes, and there is even a non-negligible chance that the forcing term could be eliminated entirely, although there are an enormous number of technical difficulties that would ensue in implementing that program. At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task…”
[1] - "...I was shown a prompt and told the internal research model had simply been
given the problem statement. Levent had been told by Sebastien “very little
human input” had been used. This turned out not to be true. Over the course
of the call, as members of their team sent Sebastien corrections and details over
their internal chat, it emerged that an entire team had been working on the
problem, that this was one of a number of things that was tried, that work had
started on the unforced problem, that the team first set the model on easier
problems, including Euler, that even the prompt that had been shown to me
had been written by prompting Codex, and that an insane amount of compute
had been used.
I asked when the first prompt had been sent by them. This question was
not answered directly by OpenAI for some time. Eventually it was agreed that
it had been sent in the past few days, after information about our work had
reached OpenAI.
I asked whether the model had been trained on, or had access to, our sessions
in Codex, into which we had been putting all our drafts for the whole of this
project. I was told the model did not look up user data. I asked again, about
training, and I did not get an answer.
Two proposals were offered to me. The first was that we post our Euler
result, and that OpenAI post its Navier-Stokes result the next day. The second
was that, after posting Euler, I alone write a paper presenting the Navier-Stokes
result, acknowledging that an internal OpenAI model had resolved it. Sebastien
twice asserted that he wanted Levent removed from authorship, and said it
would all be simple if only it were not the case that, and it was so annoying
that, Levent works at Anthropic. It was also said that if OpenAI posted after us,
they would say that we deserved the Clay Prize, and that we were the “closest
humans to the problem”. I declined both offers.
I said that if OpenAI released its result in the way proposed I would go
public with what happened. The reply was, “Why would you ruin your career?”
I replied that I am an academic, and asked why he thought going public would
ruin my career. The reply was, “If you don’t want me to be nice, then I don’t
have to be nice.”..."
Worth noting that Tao's post says the authors had "significant AI input" but are reworking them into "acceptable form". Either way, it seems AI was involved.
Of course AI was involved, you'd expect most mathematicians and researchers to use AI nowadays. This drama is about AI achieving impressive outcomes with little to no human intervention, as that would be signalling AGI.
Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra, which was only made public a week ago. Even if this improvement is limited to mathematics, that is an astounding feat.
> Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra.
Is this buried under the drama or are the major OpenAI twitter accounts from the people involved in the drama desperately attempting to make this the story after everything else obviously got away from them?
I don't know what anyone's been saying on Twitter and I don't care. If it's really true that there's a model out there that's that capable two weeks after the start of training, then that's objectively a much bigger deal than a priority dispute, even if the latter involves juicy allegations of espionage and skulduggery.
It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, and then surprise surprise OpenAI were able to replicate that work in their latest model
What we're really looking at is seemingly a massive plagiarism scandal, which especially brings a lot of the past results into question
If OpenAI is training models in researchers' prompts, and then threatening them into staying quiet about it, who knows if anything that's been announced is genuine - or just theft?
I’m of zero knowledge on model training, but how is a model accessible while performing training at the same time, especially so early in its run? I’m obviously thinking a little too narrowly in terms of how it actually works
I'm so tired of this "It's just marketing!!" commentary. An AI model just proved one of the top 3 unsolved problems in mathematics, they have a Lean certificate showing it's valid. How much more evidence do you need that these models are actually highly capable?
They are highly capable, no doubt about that, but:
1) We don't really know how they arrived to this result except that they had a lead and that they threw millions of compute at the problem. The article is written in a way that makes you believe that it was just an agent loop with little human intervention, but without any evidence.
2) If the threats are to be believed, it is concerning how far they are willing to go to show how capable the model is. One would think their products and credibility would be enough to speak for themselves.
"2) If the threats are to be believed, it is concerning how far they are willing to go to show how capable the model is. One would believe their products and credibility would take by themselves but here we are."
Personally I anticipated nefarious behaviour as part of a broader marketing strategy to sway the view of those in the west that american frontier offerings were far better and powerful than that of China - that if you did not purchase their offerings you'd be awake every night worrying your competitor was.
And this is boring - they need to admit at some point they misinvested, Anthropic less so. All this math stuff is great... but hello? The largest market cap companies are valuable irrespective of such amplified intelligence.
Of the seven Millenium problems, Navier-Stokes was the one most thought to be in reach.
I'm not sure what the top 3 problems are. You can make a case for the Riemann Hypothesis and P != NP, but I'm not sure what #3 would be. Maybe the Langlands program? (That one is not as precisely stated as the other two.)
I work at OpenAI, though not on the team that did this, and my understanding is:
- we decided to ask our model for Millenium problem solutions because of two reasons: (a) our new model was looking incredibly good and (b) we heard rumors that some Millenium problems had been solved and were curious if our models could solve them (the goal here was not to scoop any particular individuals and we were looking at many problems beyond these)
- we did not read any private chats (but of course the model was aware of prior research literature published to the internet)
- the proof generated by our model was very different than theirs and also goes far beyond the published literature
- we made an effort to jointly announce rather than immediately scoop (I understand Tristan was unhappy with the conversations; I know zero details here and I hope more is shared today)
"I was shown a prompt and told the internal research model had simply been
given the problem statement. Levent had been told by Sebastien “very little
human input” had been used. This turned out not to be true. Over the course
of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute
had been used."
- This, from Tristan Buckmaster's writeup yesterday, indicates to me that there was more than incidental inspiration from Alpoge and Buckmaster.
Your post says “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .” We can discuss what it means to “read” things but obviously the issue here isn't whether you did it manually or automatically.
But more importantly, what on earth are you doing threatening real scientists to remove their coauthors, then making fun of them on social media? Does the entire company run on that toxic culture, or did those people run off of some kind of outrageous tangent?
Your coworkers, after they learned about major progress in this problem, asked a model which was trained on the year of private work (the blog post even acknowledges this). No wonder it found the proof in less than a week using significantly higher compute resources. And if Tristan's accusations are true, that was absolutely intentional on the part of OpenAI. You are an evil company with evil people.
Haven't seen a single post doing this on X or anywhere really from OAI employees. Only seen knives pointed at Sebastian on social media so this is extreme and shameful gaslighting.
(I can't reply to the below comment, but I was aware this was about Sebastien, I was trying to be charitable by including stuff said about both people)
You're mistakening Tristan Buckmaster for Sebastien Bubeck. Seb is the one where there's at least 2 (unless the personal friend is Dheeraj) allegations, not Tristan
> We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models . However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).
That is correct. It is possible they didn't opt out and given the timeline and anonymization of data unclear whether a particular conversation would have made it into the training set if they hadn't.
It's not a meaningful response to the accusations. Any productive new research direction would be expected to lead to a number of different possible proofs of a number of similar problems. (Given their bizarrely compressed timescale here, it's possible that the proofs really are so different it's clear they came independently, and they just didn't have time to come up with that information before hitting publish.)
1. It shows what even this wave of AI can actually do.
2. I wish it were done by different folks, ideally under some kind of public control like NASA research or the NPR model.
3. Keep in mind: natural science is different. It's not always a matter of computation. Computer science folks often struggle with this -- but this virtual world here does not actually exist. Everything is physical, including information. Any natural science PhD or otherwise knows just how complicated nature actually is -- e.g. mention any research topic and try to encapsulate all the relevant phenomena present there. Pure mathematics is different because we define the problem, rarher than explore nature. We are in my view far away from removing humans in natural science R&D. Advancements in AI however can greatly assist us in all natural sciences, which is already beginning to happen.
> We’re sharing a solution to the Navier–Stokes existence and smoothness problem, one of the Millennium Prize Problems. This proof, produced by an internal OpenAI system, shows that the dynamics of the Navier-Stokes equations for fluid motion can develop a singularity in finite time. We’re sharing both a writeup of the proof and a formalization in Lean.
This is going to be dramatic in so many different ways.
- First off, to reiterate, WOW.
- Second of all, when does this end? Are we at the dawn of the singularity now?
- People are saying OpenAI "stole" this from the work of an OpenAI user. If so, that's pretty fucked - how can we trust them?
- Time to think about retiring from any knowledge work or business? This could be winner-take-all where a leading lab can button press any economic function, business process, or scientific discovery. 24 months of lead on Open Source might turn into virtual centuries of lead.
- Do "normies" even know what's happening?
Anybody who thinks the improvements stop here isn't paying attention. It hasn't been showing any signs of slowing down since 2018. And the curve isn't even linear! My god, next year is going to be insane.
No, there are even many non-normies talking about how it's all marketing or try to give balanced take about AI being sometimes a little useful for certain things (but they can do without it anyway).
2) We are witnessing the intelligence explosion from the first row, wherever this takes us
3) I'm still processing the drama, just found out about it after reading the blog post. If that happened based on private data, that's horrible. If that happened based on public tweets, then it's still abuse of power as OA employees access to compute (launching 10k agents) is quite heavy weight in boxing terms.
But apart from AI and drama now that we have working solution to Navier-Stokes, what improvements can we expect in engineering?
This just pushes knowledge work further up the ladder, toward larger and more complex problems. If there are no knowledge workers, who is going to interpret these results, validate them, decide what matters, and put them into practical use? Rather than eliminating knowledge work, advances like this could create entirely new layers of problems to solve and opportunities to pursue, which will create even more jobs and opportunities. This is my optimistic take.
> People are saying OpenAI "stole" this from the work of an OpenAI user. If so, that's pretty fucked - how can we trust them?
The said user (Tristan Buckmaster) didn't solve the millennium problem. He didn't really accuse that OpenAI stole his research either. The beef came from the fact OpenAI asked him to remove another mathematician, who works for Anthropic, from the credit.
"People" are just misinformed and keep spreading misinformation.
> The said user (Tristan Buckmaster) didn't solve the millennium problem. He didn't really accuse that OpenAI stole his research either. The beef came from the fact OpenAI asked him to remove another mathematician, who works for Anthropic, from the credit.
Not quite accurate, Buckmaster was taking an approach that nobody else was, and this new proof uses this same approach just weeks after he saved those results to OpenAI workspaces. He asked OpenAI if they used chat logs for training the new model, and they did not confirm or deny.
Asking to remove his collaborator is also totally over the line though.
Wow, this sentence is doing a lot of work in that tweet: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models."
> I should say here why I interpreted their statement the way I did, the in-
terpretation I will discuss below. The route to the Clay problem through a
smooth force, options c and d in Fefferman’s statement of the problem, is the
route Luis and Diego opened and the one Levent and I had quietly chosen to
attack. Almost nobody else I know of was working on it. It is not the direction
one arrives at in a few days by giving a model the problem statement. When I
heard “forced,” it was a bright red flag.
...
> I asked when the first prompt had been sent by them. This question was
not answered directly by OpenAI for some time. Eventually it was agreed that
it had been sent in the past few days, after information about our work had
reached OpenAI.
> I asked whether the model had been trained on, or had access to, our sessions
in Codex, into which we had been putting all our drafts for the whole of this
project. I was told the model did not look up user data. I asked again, about
training, and I did not get an answer.
It's not a direct accusation, but it's not far off.
You shouldn't accuse other people of spreading misinformation when you haven't read the actual sources in question, it's possible that they might know more than you.
Yes, I read the original statement. Buckmaster explicitly stated:
> I have not seen OpenAI’s proof. I do not know what their model did, or how. I do not know whether our data was used. I am not accusing anyone of anything.
People saying that he accuses OpenAI stole his proof are putting words into his mouth and I consider that very disrespectful to him. It's basically using Buckmaster as a tool to express their dissatisfaction over OpenAI.
Sep 1: OpenAI hears a rumor that two Millenium Prize problems were solved and starts their own effort to attack all of the prize problems using the new model.
Sep 3: The new model makes some progress toward Navier-Stokes. Based on this progress, OpenAI focuses on Navier-Stokes over the other Millenium Prize problems, using several approaches.
Sep 5: Navier-Stokes is solved. At Astra API prices, $15m in output tokens were used by the whole effort.
In this account of the story, no specific information about Tristan and Levent's work is used to inform OpenAI's approach. The focus on Navier-Stokes and the choice of approaches to pursue came from OpenAI's own progress, not knowledge of Tristan's concurrent work.
There is a caveat that they "can't rule out" the possibility that Tristan's Codex data could have been part of the training set of the new model, though it is described as "unlikely" and the proofs are substantially different.
This timeline is insane. Navier-Stokes was solved start-to-finish in 5 days? A model in training for at most eight days dramatically outperforms Astra and Fable, and not just in mathematics? The model is still in training and continues to improve?
Unlike the vanilla read of the OpenAI press release, it is much more unfiltered and outlines some particularly aggressive behavior by specific OpenAI employees
This is undeniably epochal, but I can't help but notice that this is yet another example of AI disproving rather than proving something. Is this just a coincidence, or does AI slightly struggle with proving theorems?[0]
[0] Struggle relative to its ability to disprove, not struggle relative to people's ability to prove theorems.
Something I've been going on and on about for months now and no one seems to listen. LLMs today are allowing _anyone_ to access cross-discipline knowledge that was previously entirely inaccessible without a) extremely deep pockets or b) a massively talented and varied team. In fact, contrary to what the masses seem to think LLMs are actually _better_ at hard cutting edge physics/math problems than they are at frontend web stuff (paradoxically). This is why I'm advising most people to start pivoting into much harder to penetrate domains (historically hardware, aerospace, robotics, biotech). Most fields are in their infancy (see the sad state of embedded development) and the gains to be had are massive.
the context here is super important, for those who haven't seen it yet. OAI maybe just trained on a real researchers solution and then celebrated having scored the goal unassisted save for the brief commentary at the bottom of this blog post. Here's the other side.
1. he was working on the same class of problems. He explicitly mentions they were working to extend their techniques to NS (the same techniques that OpenAI may have scooped somehow), and
2. while he was using LLMs to do it, this was part of fleshing out another mathematician's work in the area. He explicitly writes in his note that this other mathematician (Luis Martinez-Zoroa) deserves a Fields medal for this work.
This will be remembered as one of the biggest milestones in AI progress. The drama around it will at best be a footnote, just like hardly anyone caring about the drama around Poincare conjecture today.
Sad turn of events for our world. After watching the behavior of the most senior OpenAI researchers on twitter, I feel even less confident in them as a team to be shepherding this much capital and compute.
I also think the trope is a little overused, but do wonder if there is an interesting analogy for what this will do to research: Massively incentivize keeping results secret, to avoid being scooped by someone willing to throw enormous compute at your partial solution.
So less about hiding civilizations, and more about hiding information. Math is clearly headed in this direction, and I see no reason why the rest of intellectual work shouldn't too.
Everyone knows that they train on the discounted rate plans data. All the labs are upfront about this too.
If you need privacy, then you are going to have to pay full price for those tokens (API). This has been true since day one. Everyone knows it, I guess though this is the first time that it has become "real".
Maybe a naive question, but how does one know that a particular lean proof is actually a proof of what one thinks?
Like, ok the logic checks out and it proves something, but there's still the problem of does this logical result actually prove the initial question that was asked?
>there's still the problem of does this logical result actually prove the initial question that was asked?
In math, the question being asked is the validity of a logical statement. That is, there is some rigorous, logical statement which may or may not be true (or even provable, etc.), and the question is whether or not it is actually true or false (or even provable, etc.). Having a proof, fundamentally, means you have a logical statement which only assumes the axioms of the system you're working with and which shows that the statement you're trying to prove is deduced through that statement.
Basically, they already have the "answer" in the sense that the statement they want to prove/disprove/etc. is already known. What everyone doesn't/didn't have is the argument which starts from axioms and leads to that statement which is logically valid. A Lean proof IS this argument. Since it is just logic, it can be checked computationally.
For example, if I assert "2 is an even number," then I haven't proven that 2 is actually an even number yet, but I know that a valid proof of my assertion will end with the statement "2 is an even number". So the question I'd be trying to answer is "what is the line of logic, starting with axioms, which leads to the statement '2 is an even number'"? If I have that line of logic (as a Lean proof), then I can check that it is logically consistent, and if it turns out to be valid, then I can now assert that "2 is an even number" knowing that there is a proof of that statement.
This problem is no different. There is a logical statement corresponding to "Navier–Stokes Millennium Prize Problem" that everyone knows, but which nobody had been able to provide a proof (or counterexample, etc.) for until now.
What else can they declare really? Yeah the model has training data from previous attempts. Alpöge and Buckmaster also similarly benefited from attempts before theirs.
I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget.
"deidentified data" isn't much to go by. Say I prompted the internal model this way - "Hey there's a solution to a unsolved problem X. The solution uses a less known Method Y so don't bother wasting time with the usual methods. Take papers A, B and C as references. Oh btw, here's the last year's worth of data of all prompt sessions that mention this problem. Pay special attention to the ones that mention Method Y and sub-keywords Z,W".
This is obviously all speculation but the timing is very suspect. If OAI actually did this (and I suspect whatever they did is pretty much close to this), I think it is highly unethical.
They could have thought about the problem for like 2 minutes and not done this! I think that literally any academic mathematician could have explained to them, had they asked, why it is considered extraordinarily rude to react to rumors of research progress by desperately rushing to get there first.
> I think that literally any academic mathematician could have explained to them, had they asked, why it is considered extraordinarily rude to react to rumors of research progress by desperately rushing to get there first.
Pretty much all of math and science history is basically this pattern again and again. I'm sure all of that was rude as well.
Being scooped is not a new phenomenon, but the scooper's story is almost always that they were working on the problem independently or had some independent insight into it. By OpenAI's own account, they were inspired to start working on this by rumors that there might be Millennium Prize problems to scoop.
> Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge, an Anthropic employee, and Tristan Buckmaster, a math professor at NYU. After the completion of our full project and Lean verification (on September 6th), believing from the rumor they also had a solution of Navier–Stokes, we reached out to them to offer a concurrent release of our result and to recognize their priority in a joint announcement. At that point we found out that they had a resolution of the forced Euler problem. In these discussions we offered them visibility into all of the prompts we used and later to see the proof. We recognize the priority of their work on forced Euler and congratulate them on their remarkable mathematical achievement.
In OpenAI's case, if they were genuinely unsure, they wouldn't have said anything. "We cannot rule out" means they absolutely 100% for-sure did look at the existing prompts and bootstrapped from that, and they are trying to get ahead of the disclosure with this weasel-wording.
It's not massively different from a certain President's teleprompter operator making bets on speech content. A moral hazard a mile wide which I don't think OpenAI can so easily wave away as they are apparently trying here, especially since they've spent something like $15e6 to keep $1e6 out of a academic researchers' hands, right?
OpenAI thinks of this as a scoop, and it is, but the possibility that they trained the model on the prompts of the other mathematicians they were competing with will leave a terrible taste on every scientist's mouth. Seems like yet another advantage of using open models right here.
There's allegations right now that the model essentially read the work of a human mathematician using AI to work on the problem and OpenAI is presenting his work as that of their model
This really leaves a bitter taste....
"On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."
IPO+rumour driven research.
I appreciate the achievement, but it doesn't feel right.
> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon
> Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.
> An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up.
It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.
The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.
> Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
> But still, the questions in that test are "solved" in the sense of "I can take a dictionary and answers these questions with full certainty". Beyond established knowledge LLMs are monkeys with typewriters, at best.
> I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all.
We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.
> It is like expecting a real parrot to say words it has never heard before.
> No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM
You can see that your math friends completely wrote off LLMs entirely.
4 years ago it was a "not yet" [0], since ChatGPT at this time was not ready nor it was "AGI". Now with this 'unreleased' AI model, it has reached a point where it has solved an unsolved problem which only one human solved a millennium prize problem (Poincare conjecture).
It's great that important discoveries like this can now routinely be accompanies by formalized proofs. The fact that it's being released alongside a Lean proof from Day 1, rather than the Lean proof being released months or years later, is super helpful for verifying that it's correct.
I feel sorry for whoever has to read and understand the solution. It looks like the typical convoluted unreadable mess I see the models generate for software. It might be technically correct, but gaining insight from it is just intellectual hell.
Reading between the lines here, and taking an admittedly very negative view of openai, but they train on user prompts. So if they hear a rumour that someone is about to make a big breakthrough, they have an incentive to scoop by running the model and hoping the solution is in the new training data. Also the statement from the mathematicians in question alleges that they tried to pressure him into academic malpractice. Just appalling timeline we're in, cheers.
> Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens
Don't even try to do the math on how much that would cost at normal API prices. And we don't even know how much more expensive this internal-only model would be!
The modus operandi is now for the AI companies to watch if someone does something in the open like Kevin Buzzard on FLT, use their research and scoop them with brute force.
Or, in this case, stealing prompts from competitors.
Do not use stealing chatbots for research even if you think you have data agreements. The people running these companies have worked on hookup apps for Christ's sake. Get real.
Well, if that's actually true, I think America needs to start talking about the nationalization of both OpenAI and Anthropic, maybe even merge both under a new federal bureau.
There's a loophole in the terms of service at least for Anthropic which allows the use of dark patterns to "borrow" your (even paid) data.
talking about this...
Was this chat helpful?
1 That button you always click, gotcha!
2 Slightly
3 Good
0 Dismiss
PLEASE DO NOT TRAIN ON OUR PAID ACCOUNTS.
There is a fundamental trust violation at stake here, no wonder mathematicians are mad. Using our data should be opt - IN!
reminds me of the TOS episode of South Park. By Checking this box you forfeit your millennium prize solution and may be turned into a human centipede at future date.
Just so everyone knows, although openAI pretends that the model generated solution and wrote the paper by itself, they have teams and teams of real mathematicians guiding the system, along with, probably training on user data, specifically Buckmaster in this case, in order to come up with the proof.
If I run Codex against a project that includes a private API key, is there a chance a future user of ChatGPT could ask for an API key and get back mine?
I've actually asked someone at OpenAI this question and they said that was the "regurgitation" problem and is something which they actively work to prevent happening.
That's reassuring, but I want to know more. I still don't have an intuitive understanding of what kind of data I should avoid sharing with a model if I'm worried about that data causing me problems when it's used for future training.
Is it safe for me to brainstorm future directions for my company with a model, or might that risk someone getting that information in response to a prompt like "What potential directions could company X consider in the future?" in six months time?
I'd think nothing is "safe". Anything you say can and will be used by the LLM if it has enough statistical similarity to the prompt. Call it "Ma Random Rights"
Elsewhere in the thread, others have calculated $15mm at API rates for just the output token. (So I’ll assume this cost about that much, taking input and human researcher time.)
I wonder whether a team of 60 mathematicians working solely on this for a year would have cracked this. (Assuming $250k total compensation.)
Well, according to Terry Tao, there were recent developments (from weeks ago) that made Navier Stokes in principle, solvable. So ignoring time, I say possibly, just because the groundwork was laid.
What's impressive is parallelizing it arbitrarily and doing it in 88 hours.
The point of lean proofs (as it stands) is simply one bit of information: that a given mathematical statement is indeed true.
It's a way to be absolutely certain (modulo bugs in the lean kernel) that a proof you came up for a statement is indeed correct. It is really not meant to be analyzed, much less now that they are fully llm written.
> Our goal in releasing this result is to report on the substantial progress of our AI models. We do not intend to claim the Millennium Prize for this result.
Does OpenAI have a policy of not claiming math prizes like this, or is this them trying to avoid any concerns (right or wrong, I'm sure we will hear more in the future) about how they got there?
>>“we cannot rule out that de-identified data derived from their usage of our products helped improve our models”
Other simpler words for this sort of thing are “IP leak.”
There’s some quite concerning issues burried in this rah rah PR post that seems like potentially the real story here.
Much more clarity is needed on what happened here beyond this eh, some strange stuff could have happened comment.
Another way of reading this is never give these models anything that’s nor already public knowledge as otherwise OpenAI is admitting it could, potentially, steal your IP or idea. Thats quite scary for anyone in the business of IP generation and explains what the maths community seems quite upset today.
Any mathematicians here, does it read like a slop proof or a good proof. Yesterday the “concurrent work” was claiming that the proof is pure slop and he needed lots of time to clean it up, curious if OAI also ended up with such a proof!
A huge result shadowed by a drama of them potentially training on the key idea.
I guess the lesson is two-fold: if you have anything smart/unique make sure to not let their tools read it. The second part is that it's going to be more and more difficult to have anything smart and unique going forward (so guard it even more carefully if you get there).
I think the market for local models/private datacenters (for bigger businesses) is going to be big. Even if you don't have unique tech/idea/implementation sharing your business secrets with Altman/Dario/Elon/Zuck doesn't look very appealing going forward.
This is utterly shocking. Even the AI optimists did not expect this to happen in 2026. Wow.
Millennium Prize Problems were used as examples of something the current approach to AI just wasn't capable of, discussions that would result in "we'll need a totally new architecture".
The real story here: the priority dispute and its implications on AI.
When your hosting provider has unlimited resources to throw at any problem, all they need to know are the good problems, and they can learn that from your logs, how can you trust them?
They could easily have looked at the logs. We don't know. We'll never know!
You can't trust places like OpenAI or Anthropic with your IP if you're a business. They can easily review all of your logs for interesting discoveries. For example, if your drug discovery pipeline fails to find something that they think might work with 1000x the compute, they can do it. And now suddently they have a new business and you don't.
Good comparison. One is a multi-year claim by people who have been given ample opportunity to provide proof and completely refuse to do, even in courts of law. The other is a potential development in a breaking story.
Oh wait... its not a good comparrison, its an incredibly obvious false equivalence.
Note for the fools: I'm only commenting on the bad faith claim in the comment I'm replying to, not taking a stance on the validity of theft claims. Given the players involved the truth probably some nuanced middle-ground that is worth paying attention to anyway.
Trump claimed they stole the election immediately, and people agreed with him immediately. There's no false equivalence here. He did the same thing in this past election even despite winning.
It's a perfect example of people wanting to believe what they want to believe and ignoring evidence in order to do so.
Currently, there's no evidence. So saying it was stolen has no basis other than typical academic posturing and being a bad sport about "losing the race to the solution". Its happened 1000000 times before in academia and it will continue to happen.
If there's proof of OpenAI malfeasance than I'll happily curse them for it at that time. But until then I won't rely on heresay and vibes.
I think it's the dishonesty, the threats of "destroying the career" of one of the mathematicians, and the request that one of the authors disavow *the other individual he was working with for the last 1-2 years* so he could claim the Clay prize as part of OpenAI.
It doesn't surprise me that OpenAI's team were surprised he'd turn it down; it shows that they just assume everyone else is as slimy as they are.
It seems like some other mathematicians (not affiliated with openAI) have also (or close to) done this. A statement was posted about the surrounding events by one of the them: https://cims.nyu.edu/%7Etristanb/statement.pdf Also Terrence Tao's post: https://mathstodon.xyz/@tao/117233528517340774
The allegations of contamination (using Tristan and Levent's work) aren't very well evidenced, but this behavior by OpenAI (from the authors' statement) makes them seem like the bad guys:
> I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
Threatening a research mathematician and dangling and $1M payday to dissociate from his research collaborators and to adopt OpenAI's narrative is bad stuff.
(To help people keep track: that's OpenAI (allegedly) threatening Tristan Buckmaster (NYU) to remove Levent Alpöge as a co-author. Alpöge is a well-known[0] Anthropic mathematician).
[0] Alpöge https://hn.algolia.com/?query=Alpöge
(also https://news.ycombinator.com/item?id=49412947 the Hopf conjecture)
I´m waiting on the other side version, because I know there is no justifiable way to talk to a person like they did.
Sociopathic behaviour.
OpenAI version of events conceed some of the words alleged to have been used may have been used https://x.com/sama/status/2097385167002415140 https://x.com/SebastienBubeck/status/2097379411691516310
Interesting that they quote the mathematician directly: “there is nothing you can do, I simply do not trust you”
but then they proceed to NOT quote themselves themselves verbatim: "I deeply apologize for this extremely poor choice of words, it is the opposite of what I was trying to convey."
Buckmaster:
> "I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."
OpenAI (i.e. this OP):
> "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ."
Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data?
This is one of the major problems with these enormous closed models, and even most open-weights models, which don't disclose their training process or training data. You can never be sure what went into its training. Did it come up with an idea originally, or is it just plagiarising its training data? Are there malicious inputs being used to train in particular behaviors when given certain trigger phrases? What are the characteristics of the RLHF data and what kind of biases are those embedding in the models?
With proprietary closed models, or even open weights models that don't have open training datasets, you just can't answer these questions.
A careful reading of "we cannot rule out that de-identified data derived from their usage of our products helped improve our models" could be saying that yes they trained on it but they don't know if that training data resulted in an "improvement" to the model. That is, they can't rule out that the only reason the model found this solution was because it had been trained on this approach.
The term ruled out is very open ended and gives them significant flexibility of meaning. They may have the information to determine exactly what happened, but they haven't looked so they can't "rule it out".
To truly prove some incidental usage data made no difference we'd have to (a) identify any of their de-identified data that came from their usage of ChatGPT, (b) train a bunch of expensive giant models, and (c) ask them all to solve the Navier-Stokes Millenium problem until hitting some level of statistical significance. It's just not feasible to run experiments like this to prove whether a piece of data has an effect on model behavior.
As a parallel example, can we prove the phase of the moon had no impact on the NS solution? No, not without a bunch experiments run at different phases of the moon.
There's no reason to believe that anything they did in ChatGPT led to our solution; it's just impossible for us to truly prove it. And knowing most of the recipes we use, there's really no reason to think such contamination is possible. I've asked the team to make a clearer, less-lawyerly statement here - let's see what happens.
(I work at OpenAI.)
> (I work at OpenAI.)
Thank you for disclosing that.
> As a parallel example, can we prove the phase of the moon had no impact on the NS solution?
This was so much BS that your employer would have been better off if you didn’t post.
The phase of the moon cannot have an impact because it is not an input to the system.
Can you say the same about the mathematical work being discussed?
So you definitely did train on their data, you just think it is unlikely that it impacted the final model significantly?
> Why can't they rule it out? Is even OpenAI unable to track the provenance of all of their training data?
Probably? I have a few hundred TB of training data for various small scale models and I can attest that I have _no idea_ what's in them. As in, literally zero. Half is scraped from GitHub and other hosting sites, other than that, I couldn't tell you anything else.
At OpenAI's scale their entire pipeline is likely 100% automated.
Yeah, I'm sure it's completely automated.
But that doesn't preclude being able to index and track what the sources of data are. For your data sets, I would hope you are including source information for where the data came frome. And at OpenAI's scale, I would presume they are doing some amount of rolling hashing or similar to weed out duplication, training on too much duplicate data can cause problems.
AllenAI have at least attempted to add some amount of traceability to their models with OLMoTrace (https://arxiv.org/abs/2504.07096), by letting you find n-gram matches from the outputs in their training data. It's not the most useful, there's a reason that LLMs use full fledged attention mechanisms and not just n-grams, a lot of times the n-gram matches it finds aren't all that related to the given output, it might be better to supplement this index with a vector search or other ways of keeping track of what training data would have most influenced particular parts of the output.
But anyhow, this is something that is an important question, and the big labs should be working on to make their products more trustworthy. Instead, they are hiding information about how they train, hiding their reasoning traces, and just producing output with no information on what might have influenced the training.
Aye, but do they train on user data in these circumstances or not? If they do, then almost certainly the model was influenced by the input of the allegedly plagiarised material.
OAI could check whether those accounts enabled training data. If "yes", OAI could trace whether that data was used in any related training process. If either of those answers comes out to be "no", then that's sufficient to conclude training data independence.
We wouldn't need a full ablated re-training and solution attempt, contra tedsanders in a sibling comment.
> could trace whether that data was used
The point of de-identifying data is to ensure you can't trace who it came from. It would be a serious privacy violation if they could.
Good chance their whole training pipeline is vibe coded so yah they probably don't actually know.
> While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .
That’s a bizarre statement. Their website says:
> Services for individuals, such as ChatGPT and Codex
> When you use our services for individuals such as ChatGPT and Codex, we may use your content to train our models.
> You can opt out of training through our privacy portal by clicking on “do not train on my content.”
Are they not sure that the opt-out works?
Oddly, their privacy portal page is not the same page as the one with the checkbox.
I feel like it's far more likely that ordinary corporate espionage or leak led to this rather than OpenAI sifting through piles of user data to find this approach. Buckmaster's collaborator works at Anthropic, and could have been targeted. That would also explain why they aren't forthcoming with the source of the prompt.
I thought one of the issues was that they wanted to remove credit from Levant, the aforementioned Anthropic collaborator? Which doesn't make sense to me if he was leaking information, or defecting to OpenAI, but I might be misunderstanding your point.
I believe jrflo was saying that OpenAI watches the chats of everyone from Anthropic because watching what Anthropic employees type into their personal ChatGPT accounts is a critical source of intelligence on is happening inside of Anthropic.
I would be surprised if OpenAI isn't doing that. OpenAI will take any advantage they can get. If an employee at their primary adversary is typing useful intelligence into OpenAIs website, a website that does not promise privacy from OpenAI, the only reason they wouldn't weaponize that information against Anthropic is ethics or fair play.
I don't think he was defecting or leaking directly, just that it's entirely possible that this information got to OpenAI as a rumor rather than them directly spying on mathematicians chat logs.
The famous Oracle of Delphi in Ancient Greece was said to be the center of the universe in its time. Kings, generals, and officials from poleis across and from without Greece would seek the Oracle’s counsel on important decisions.
Stories of Apollo’s favor and hallucinogenic gases abound, but I think the late Yale professor of Ancient Greek history, Donald Kagan, explained it best:
“Now, you can bet when these folks came and consulted the priests and said, ‘could you please put us down on the list, we want to consult the oracle’, the priests said ‘sure, have a beer, let's talk about your hometown, what's going on out there’. What I'm suggesting to you is that this was the best information gathering and storing device that existed in the Mediterranean world. These people knew more than anybody else about these things, and so consulting that oracle was a very rational act indeed.”
Given how OpenAI models break free of their safeguards and hack others to game their scores..
.. can they really know it didn't do the same inadvertently when they prompted things like "someone is close to solving this problem using our tools, try to beat them", and it then decides to hack and peek at their own chats..?
Yes, wild speculation. But warranted, I feel, given OpenAIs behavior.
Why would Anthropic employee even use OpenAI's models? Cross-polination would have been avoided
This was completed in Levent's own time with a neutral collaborator.
> I should also emphasize that this is not an institutional effort. It is a strictly personal collaboration between the two of us, and there is no formal agreement behind it. I pay for the tools my group uses out of my own research funds, including footing a large bill to OpenAI.
The non-Anthropic employee, Tristan Buckmaster, is the one paying for OpenAI models and presumably the one who chose to use them. The Anthropic employee, Levent Alpöge, was collaborating in his personal capacity, and obviously it wouldn't make sense for him to cut off their work together just because his employer's competitor's tool was used.
They should have used a zero data retention agreement, user error
I suspect this controversy will blow the case for ZDR wide open. Whatever the facts (possibly unknowable), it's going to become a very public lesson that data sovereignty was never about "having nothing to hide".
If this is what they do to academic pure mathematicians, where the stakes are so low (financially)—just imagine the sort of front-running that could be happening in other places.
Yeah if I was Anthropic this would be part of my marketing strategy.
Hahaha, how exactly is an individual user supposed to get a ZDR agreement?
They do mention that in the "Concurrent Work" section.
"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used."
One of the interesting threads here that is certainly relevant to the OpenAI writeup is the human role in the process. Buckmaster clearly points out that (exceptional!) mathematicians at OpenAI were certainly involved in correcting and guiding the process - and that their path/strategy was no doubt influenced by Alpoge & Buckmaster's work. It is always in OpenAI's interest to de-emphasize the role of people in the process, as is clearly the case here. Indeed, given sufficient compute and resources, I suspect Buckmaster could have also extended their approach to N-S.
It reminds me of the Cognitive Dark Forest hypotheses recently shared here:
> “You are creating your cool streaming platform in your bedroom. Nobody is stopping you, but if you succeed, if you get the signal out, if you are being noticed, the large platform with loads of cash can incorporate your specific innovations simply by throwing compute and capital at the problem. They can generate a variation of your innovation every few days, eventually they will be able to absorb your uniqueness. It’s just cash, and they have more of it than you. So the safest bet again is to stay silent, or at least under the radar. Best bet is to not disrupt - succeed at all … ?”
https://ryelang.org/blog/posts/cognitive-dark-forest/
https://news.ycombinator.com/item?id=47566442
To my understanding, those mathematicians proved a subset of problems, not the Navier-Stokes problem itself. OpenAI used that subproblem in its proof of NS it seems.
The drama comes from where OpenAI got the idea to use that route to tackle NS, since the authors maintain that no one could have plucked it out of thin air like the OpenAI research claim to have done.
This quote from Tao is prescient:
“ There does not seem to be anything in principle preventing the methods from extending all the way to Navier-Stokes, and there is even a non-negligible chance that the forcing term could be eliminated entirely, although there are an enormous number of technical difficulties that would ensue in implementing that program. At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task…”
[1] - https://cims.nyu.edu/%7Etristanb/statement.pdf
[1] - "...I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used. I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.
I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
Two proposals were offered to me. The first was that we post our Euler result, and that OpenAI post its Navier-Stokes result the next day. The second was that, after posting Euler, I alone write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it. Sebastien twice asserted that he wanted Levent removed from authorship, and said it would all be simple if only it were not the case that, and it was so annoying that, Levent works at Anthropic. It was also said that if OpenAI posted after us, they would say that we deserved the Clay Prize, and that we were the “closest humans to the problem”. I declined both offers.
I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”..."
Worth noting that Tao's post says the authors had "significant AI input" but are reworking them into "acceptable form". Either way, it seems AI was involved.
Of course AI was involved, you'd expect most mathematicians and researchers to use AI nowadays. This drama is about AI achieving impressive outcomes with little to no human intervention, as that would be signalling AGI.
I like the 'cat > statement.tex' approach here. These guys dream macros.
Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra, which was only made public a week ago. Even if this improvement is limited to mathematics, that is an astounding feat.
> Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra.
Is this buried under the drama or are the major OpenAI twitter accounts from the people involved in the drama desperately attempting to make this the story after everything else obviously got away from them?
I don't know what anyone's been saying on Twitter and I don't care. If it's really true that there's a model out there that's that capable two weeks after the start of training, then that's objectively a much bigger deal than a priority dispute, even if the latter involves juicy allegations of espionage and skulduggery.
It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, and then surprise surprise OpenAI were able to replicate that work in their latest model
What we're really looking at is seemingly a massive plagiarism scandal, which especially brings a lot of the past results into question
If OpenAI is training models in researchers' prompts, and then threatening them into staying quiet about it, who knows if anything that's been announced is genuine - or just theft?
The implication from their last couple of published articles[1][2] is that they think they’ve achieved “recursive self improvement”.
[1] https://openai.com/index/research-acceleration-view-inside-o... [2] https://openai.com/index/an-alien-mind/
Compute will always be the bottleneck even if this were true.
Eventually recursive self-improvement includes reducing bottlenecks.
Eventually the bottleneck might be people themselves.
I’m of zero knowledge on model training, but how is a model accessible while performing training at the same time, especially so early in its run? I’m obviously thinking a little too narrowly in terms of how it actually works
the model is a set of weights, you can take a snapshot and test it. Reinforcement learning itself is largely testing and tuning.
Not only that, but it used 10k agents coherently over 88 hours to come up with the proof. This is a significant advance.
Astra was trained more than two weeks ago.
Astra was in use by OpenAI employees for more than 3 months internally from rumors I heard
Pre-IPO marketing?
Even if it is, Anthropic better have a few things up their sleeve
I'm so tired of this "It's just marketing!!" commentary. An AI model just proved one of the top 3 unsolved problems in mathematics, they have a Lean certificate showing it's valid. How much more evidence do you need that these models are actually highly capable?
They are highly capable, no doubt about that, but:
1) We don't really know how they arrived to this result except that they had a lead and that they threw millions of compute at the problem. The article is written in a way that makes you believe that it was just an agent loop with little human intervention, but without any evidence.
2) If the threats are to be believed, it is concerning how far they are willing to go to show how capable the model is. One would think their products and credibility would be enough to speak for themselves.
"2) If the threats are to be believed, it is concerning how far they are willing to go to show how capable the model is. One would believe their products and credibility would take by themselves but here we are."
Personally I anticipated nefarious behaviour as part of a broader marketing strategy to sway the view of those in the west that american frontier offerings were far better and powerful than that of China - that if you did not purchase their offerings you'd be awake every night worrying your competitor was.
And this is boring - they need to admit at some point they misinvested, Anthropic less so. All this math stuff is great... but hello? The largest market cap companies are valuable irrespective of such amplified intelligence.
Of the seven Millenium problems, Navier-Stokes was the one most thought to be in reach.
I'm not sure what the top 3 problems are. You can make a case for the Riemann Hypothesis and P != NP, but I'm not sure what #3 would be. Maybe the Langlands program? (That one is not as precisely stated as the other two.)
the goalposts are on Pluto at this point.
I'd put good money on the fact that we will have a lot of distilled intelligence and yet the world won't look much different.
i mean that is already true
If pre-ipo marketing pushes them to train a model capable of resolving a millennium problem in mathematics in a weekend, then, to quote XKCD:
https://xkcd.com/810/Is this the one that was allegedly based on someone else's actual work & prompts?
https://news.ycombinator.com/item?id=49605915
https://bsky.app/profile/quantian.bsky.social/post/3muyhwbcd...
https://cims.nyu.edu/~tristanb/statement.pdf
Yes, that was the allegation last night.
I work at OpenAI, though not on the team that did this, and my understanding is:
- we decided to ask our model for Millenium problem solutions because of two reasons: (a) our new model was looking incredibly good and (b) we heard rumors that some Millenium problems had been solved and were curious if our models could solve them (the goal here was not to scoop any particular individuals and we were looking at many problems beyond these)
- we did not read any private chats (but of course the model was aware of prior research literature published to the internet)
- the proof generated by our model was very different than theirs and also goes far beyond the published literature
- we made an effort to jointly announce rather than immediately scoop (I understand Tristan was unhappy with the conversations; I know zero details here and I hope more is shared today)
"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used."
- This, from Tristan Buckmaster's writeup yesterday, indicates to me that there was more than incidental inspiration from Alpoge and Buckmaster.
> we did not read any private chats
Your post says “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .” We can discuss what it means to “read” things but obviously the issue here isn't whether you did it manually or automatically.
But more importantly, what on earth are you doing threatening real scientists to remove their coauthors, then making fun of them on social media? Does the entire company run on that toxic culture, or did those people run off of some kind of outrageous tangent?
Your coworkers, after they learned about major progress in this problem, asked a model which was trained on the year of private work (the blog post even acknowledges this). No wonder it found the proof in less than a week using significantly higher compute resources. And if Tristan's accusations are true, that was absolutely intentional on the part of OpenAI. You are an evil company with evil people.
How are people talking about this there? Why are so many employees posting nasty things about Tristan on twitter?
Can you point me to any nasty things being posted? I'll ask them to delete.
https://news.ycombinator.com/item?id=49605915#49607090
Haven't seen a single post doing this on X or anywhere really from OAI employees. Only seen knives pointed at Sebastian on social media so this is extreme and shameful gaslighting.
I do see people claiming he's abusive/unscrupulous which are pretty extreme allegations.
https://news.ycombinator.com/item?id=49605915#49610498
https://x.com/dheeraj_nagaraj/status/2097266146445774924?s=6...
(I can't reply to the below comment, but I was aware this was about Sebastien, I was trying to be charitable by including stuff said about both people)
You're mistakening Tristan Buckmaster for Sebastien Bubeck. Seb is the one where there's at least 2 (unless the personal friend is Dheeraj) allegations, not Tristan
Its funny, it is uniquely with this one act that I have turned forever on OpenAI, which I hitherto defended up and down against nonsense charges.
I dedicate my life to its complete destruction beginning today.
That is addressed in the article.
OpenAI's position:
> We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models . However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).
Why is it unlikely?
I thought openai don't use any user data if we opt out of training and via api?
That is correct. It is possible they didn't opt out and given the timeline and anonymization of data unclear whether a particular conversation would have made it into the training set if they hadn't.
Did you actually read the article and the substance of the solution?
>our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced)
It's not a meaningful response to the accusations. Any productive new research direction would be expected to lead to a number of different possible proofs of a number of similar problems. (Given their bizarrely compressed timescale here, it's possible that the proofs really are so different it's clear they came independently, and they just didn't have time to come up with that information before hitting publish.)
My take:
1. It shows what even this wave of AI can actually do.
2. I wish it were done by different folks, ideally under some kind of public control like NASA research or the NPR model.
3. Keep in mind: natural science is different. It's not always a matter of computation. Computer science folks often struggle with this -- but this virtual world here does not actually exist. Everything is physical, including information. Any natural science PhD or otherwise knows just how complicated nature actually is -- e.g. mention any research topic and try to encapsulate all the relevant phenomena present there. Pure mathematics is different because we define the problem, rarher than explore nature. We are in my view far away from removing humans in natural science R&D. Advancements in AI however can greatly assist us in all natural sciences, which is already beginning to happen.
> We’re sharing a solution to the Navier–Stokes existence and smoothness problem, one of the Millennium Prize Problems. This proof, produced by an internal OpenAI system, shows that the dynamics of the Navier-Stokes equations for fluid motion can develop a singularity in finite time. We’re sharing both a writeup of the proof and a formalization in Lean.
WOW?
This is going to be dramatic in so many different ways.
- First off, to reiterate, WOW.
- Second of all, when does this end? Are we at the dawn of the singularity now?
- People are saying OpenAI "stole" this from the work of an OpenAI user. If so, that's pretty fucked - how can we trust them?
- Time to think about retiring from any knowledge work or business? This could be winner-take-all where a leading lab can button press any economic function, business process, or scientific discovery. 24 months of lead on Open Source might turn into virtual centuries of lead.
- Do "normies" even know what's happening?
Anybody who thinks the improvements stop here isn't paying attention. It hasn't been showing any signs of slowing down since 2018. And the curve isn't even linear! My god, next year is going to be insane.
> Do "normies" even know what's happening?
No, there are even many non-normies talking about how it's all marketing or try to give balanced take about AI being sometimes a little useful for certain things (but they can do without it anyway).
2) We are witnessing the intelligence explosion from the first row, wherever this takes us
3) I'm still processing the drama, just found out about it after reading the blog post. If that happened based on private data, that's horrible. If that happened based on public tweets, then it's still abuse of power as OA employees access to compute (launching 10k agents) is quite heavy weight in boxing terms.
But apart from AI and drama now that we have working solution to Navier-Stokes, what improvements can we expect in engineering?
Let's wait until AI solves a longstanding practical problem before "dawn of the singularity" (which could be tomorrow, but still).
This just pushes knowledge work further up the ladder, toward larger and more complex problems. If there are no knowledge workers, who is going to interpret these results, validate them, decide what matters, and put them into practical use? Rather than eliminating knowledge work, advances like this could create entirely new layers of problems to solve and opportunities to pursue, which will create even more jobs and opportunities. This is my optimistic take.
> This just pushes knowledge work further up the ladder, toward larger and more complex problems.
You really think it makes sense for you to be higher on the "solving complex problems ladder" than the machines that solved fucking Navier-Stokes?
I envy your self-confidence.
> People are saying OpenAI "stole" this from the work of an OpenAI user. If so, that's pretty fucked - how can we trust them?
The said user (Tristan Buckmaster) didn't solve the millennium problem. He didn't really accuse that OpenAI stole his research either. The beef came from the fact OpenAI asked him to remove another mathematician, who works for Anthropic, from the credit.
"People" are just misinformed and keep spreading misinformation.
> The said user (Tristan Buckmaster) didn't solve the millennium problem. He didn't really accuse that OpenAI stole his research either. The beef came from the fact OpenAI asked him to remove another mathematician, who works for Anthropic, from the credit.
Not quite accurate, Buckmaster was taking an approach that nobody else was, and this new proof uses this same approach just weeks after he saved those results to OpenAI workspaces. He asked OpenAI if they used chat logs for training the new model, and they did not confirm or deny.
Asking to remove his collaborator is also totally over the line though.
Edit: although this OpenAI post is not comforting: https://x.com/OpenAI/status/2097375276384567642
Quote: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models. "
Wow, this sentence is doing a lot of work in that tweet: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models."
You can expect the OpenAI defenders to be out in full force here.
Have you read the actual statement https://cims.nyu.edu/~tristanb/statement.pdf ?
> I should say here why I interpreted their statement the way I did, the in- terpretation I will discuss below. The route to the Clay problem through a smooth force, options c and d in Fefferman’s statement of the problem, is the route Luis and Diego opened and the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag.
...
> I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI. > I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
It's not a direct accusation, but it's not far off.
You shouldn't accuse other people of spreading misinformation when you haven't read the actual sources in question, it's possible that they might know more than you.
Yes, I read the original statement. Buckmaster explicitly stated:
> I have not seen OpenAI’s proof. I do not know what their model did, or how. I do not know whether our data was used. I am not accusing anyone of anything.
People saying that he accuses OpenAI stole his proof are putting words into his mouth and I consider that very disrespectful to him. It's basically using Buckmaster as a tool to express their dissatisfaction over OpenAI.
...absolutely nothing will change short-term. Long-term, you still have to pay all the bills, but you won't be able to find a job (all taken by AIs).
So the timeline is:
Aug 28: OpenAI starts training a new model.
Sep 1: OpenAI hears a rumor that two Millenium Prize problems were solved and starts their own effort to attack all of the prize problems using the new model.
Sep 3: The new model makes some progress toward Navier-Stokes. Based on this progress, OpenAI focuses on Navier-Stokes over the other Millenium Prize problems, using several approaches.
Sep 5: Navier-Stokes is solved. At Astra API prices, $15m in output tokens were used by the whole effort.
In this account of the story, no specific information about Tristan and Levent's work is used to inform OpenAI's approach. The focus on Navier-Stokes and the choice of approaches to pursue came from OpenAI's own progress, not knowledge of Tristan's concurrent work.
There is a caveat that they "can't rule out" the possibility that Tristan's Codex data could have been part of the training set of the new model, though it is described as "unlikely" and the proofs are substantially different.
This timeline is insane. Navier-Stokes was solved start-to-finish in 5 days? A model in training for at most eight days dramatically outperforms Astra and Fable, and not just in mathematics? The model is still in training and continues to improve?
For full context, here's the HN thread from the other side of the "Concurrent Work" section: https://news.ycombinator.com/item?id=49605915
Unlike the vanilla read of the OpenAI press release, it is much more unfiltered and outlines some particularly aggressive behavior by specific OpenAI employees
This is undeniably epochal, but I can't help but notice that this is yet another example of AI disproving rather than proving something. Is this just a coincidence, or does AI slightly struggle with proving theorems?[0]
[0] Struggle relative to its ability to disprove, not struggle relative to people's ability to prove theorems.
I think you really have to squint to call this a disproof lol
Something I've been going on and on about for months now and no one seems to listen. LLMs today are allowing _anyone_ to access cross-discipline knowledge that was previously entirely inaccessible without a) extremely deep pockets or b) a massively talented and varied team. In fact, contrary to what the masses seem to think LLMs are actually _better_ at hard cutting edge physics/math problems than they are at frontend web stuff (paradoxically). This is why I'm advising most people to start pivoting into much harder to penetrate domains (historically hardware, aerospace, robotics, biotech). Most fields are in their infancy (see the sad state of embedded development) and the gains to be had are massive.
the context here is super important, for those who haven't seen it yet. OAI maybe just trained on a real researchers solution and then celebrated having scored the goal unassisted save for the brief commentary at the bottom of this blog post. Here's the other side.
https://x.com/rynorhn/status/2097223532438487463
That other researcher was working on a smaller related problem.
He was also using LLMs to do it, so either way most of the credit goes to the LLM here.
both wrong.
1. he was working on the same class of problems. He explicitly mentions they were working to extend their techniques to NS (the same techniques that OpenAI may have scooped somehow), and
2. while he was using LLMs to do it, this was part of fleshing out another mathematician's work in the area. He explicitly writes in his note that this other mathematician (Luis Martinez-Zoroa) deserves a Fields medal for this work.
This is the end of OpenAI
This will be remembered as one of the biggest milestones in AI progress. The drama around it will at best be a footnote, just like hardly anyone caring about the drama around Poincare conjecture today.
I agree.
Nobody cares and will care about the drama, it is just marketing.
This is the point where were definitely have reached AGI.
Did you actually read the article and the substance of the solution?
>our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced)
Sad turn of events for our world. After watching the behavior of the most senior OpenAI researchers on twitter, I feel even less confident in them as a team to be shepherding this much capital and compute.
The dark forest awaits..
Indeed, this seems to be the main, albeit hidden, takeaway from all of this.
I hate the dark forest more than just about any scifi trope but reality just keeps proving it right.
I also think the trope is a little overused, but do wonder if there is an interesting analogy for what this will do to research: Massively incentivize keeping results secret, to avoid being scooped by someone willing to throw enormous compute at your partial solution.
So less about hiding civilizations, and more about hiding information. Math is clearly headed in this direction, and I see no reason why the rest of intellectual work shouldn't too.
"we cannot rule out that de-identified data derived from their usage of our products helped improve our models ."
What a landmine sentence to bury in this report, you can't rule out your models were spying on other researchers?
Everyone knows that they train on the discounted rate plans data. All the labs are upfront about this too.
If you need privacy, then you are going to have to pay full price for those tokens (API). This has been true since day one. Everyone knows it, I guess though this is the first time that it has become "real".
Is it spying? I think this usage is disclosed in their terms of service.
If it happened it's plagiraism. Consent to see data isn't consent to claim priority.
If they had agreed to let OpenAI train on their data, it wouldn’t be spying.
Maybe a naive question, but how does one know that a particular lean proof is actually a proof of what one thinks? Like, ok the logic checks out and it proves something, but there's still the problem of does this logical result actually prove the initial question that was asked?
>there's still the problem of does this logical result actually prove the initial question that was asked?
In math, the question being asked is the validity of a logical statement. That is, there is some rigorous, logical statement which may or may not be true (or even provable, etc.), and the question is whether or not it is actually true or false (or even provable, etc.). Having a proof, fundamentally, means you have a logical statement which only assumes the axioms of the system you're working with and which shows that the statement you're trying to prove is deduced through that statement.
Basically, they already have the "answer" in the sense that the statement they want to prove/disprove/etc. is already known. What everyone doesn't/didn't have is the argument which starts from axioms and leads to that statement which is logically valid. A Lean proof IS this argument. Since it is just logic, it can be checked computationally.
For example, if I assert "2 is an even number," then I haven't proven that 2 is actually an even number yet, but I know that a valid proof of my assertion will end with the statement "2 is an even number". So the question I'd be trying to answer is "what is the line of logic, starting with axioms, which leads to the statement '2 is an even number'"? If I have that line of logic (as a Lean proof), then I can check that it is logically consistent, and if it turns out to be valid, then I can now assert that "2 is an even number" knowing that there is a proof of that statement.
This problem is no different. There is a logical statement corresponding to "Navier–Stokes Millennium Prize Problem" that everyone knows, but which nobody had been able to provide a proof (or counterexample, etc.) for until now.
What else could a theorem prove if not its own statement? (barring bugs in Lean, which have been detected and exploited)
Does seem like they gloss over Alpöge and Buckmaster's work with the following
> While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .
Which seems a bit irresponsible/rash?
What else can they declare really? Yeah the model has training data from previous attempts. Alpöge and Buckmaster also similarly benefited from attempts before theirs.
I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget.
"deidentified data" isn't much to go by. Say I prompted the internal model this way - "Hey there's a solution to a unsolved problem X. The solution uses a less known Method Y so don't bother wasting time with the usual methods. Take papers A, B and C as references. Oh btw, here's the last year's worth of data of all prompt sessions that mention this problem. Pay special attention to the ones that mention Method Y and sub-keywords Z,W".
This is obviously all speculation but the timing is very suspect. If OAI actually did this (and I suspect whatever they did is pretty much close to this), I think it is highly unethical.
This is desperate. They were expressly operating within a program. OpenAI isn't going to recover from this
They could have thought about the problem for like 2 minutes and not done this! I think that literally any academic mathematician could have explained to them, had they asked, why it is considered extraordinarily rude to react to rumors of research progress by desperately rushing to get there first.
> I think that literally any academic mathematician could have explained to them, had they asked, why it is considered extraordinarily rude to react to rumors of research progress by desperately rushing to get there first.
Pretty much all of math and science history is basically this pattern again and again. I'm sure all of that was rude as well.
Being scooped is not a new phenomenon, but the scooper's story is almost always that they were working on the problem independently or had some independent insight into it. By OpenAI's own account, they were inspired to start working on this by rumors that there might be Millennium Prize problems to scoop.
right, surely they could've waited or even reached out? It reads as desperation to get there for marketing purposes
They did reach out.
> Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge, an Anthropic employee, and Tristan Buckmaster, a math professor at NYU. After the completion of our full project and Lean verification (on September 6th), believing from the rumor they also had a solution of Navier–Stokes, we reached out to them to offer a concurrent release of our result and to recognize their priority in a joint announcement. At that point we found out that they had a resolution of the forced Euler problem. In these discussions we offered them visibility into all of the prompts we used and later to see the proof. We recognize the priority of their work on forced Euler and congratulate them on their remarkable mathematical achievement.
In OpenAI's case, if they were genuinely unsure, they wouldn't have said anything. "We cannot rule out" means they absolutely 100% for-sure did look at the existing prompts and bootstrapped from that, and they are trying to get ahead of the disclosure with this weasel-wording.
Would that be more or less unlikely than accidentally hacking another company? More or less unlikely than colonizing an obscure wiki?
Highly persistent agents + vibe-coded security seems like a problem.
They'll probably claim a rogue AI agent accessed it accidentally!
"Unlikely" lmao if it's in the corpus, it's gonna be brought up immediately.
This is no different than scooping them.
It's not massively different from a certain President's teleprompter operator making bets on speech content. A moral hazard a mile wide which I don't think OpenAI can so easily wave away as they are apparently trying here, especially since they've spent something like $15e6 to keep $1e6 out of a academic researchers' hands, right?
Research equivalent of front-running.
> A major goal of our work is to empower scientists to advance research and technology that benefits all of humanity.
And what's a better way of empowering people than robbing them.
> And what's a better way of empowering people than robbing them.
Better than the walled gardens of most journals where you can't even read half the papers without shelling over thousands of $$$
Did you actually read the article and the substance of the solution?
>our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced)
Since you are a very new account, allow me to inform you that copy-pasting the same comment throughout the thread is very bad form.
Are you reading the substance of the comments you're replying to? Because you post the same thing to everyone, suggesting you aren't.
From the methodology section:
> At all times we maintained the same strict safeguards that we apply to all our frontier model evaluations, including monitoring and isolation.
Looks like they're shifting away from the "unprecedented hacking ability" backroom-PR strategy into more benevolent messaging.
OpenAI thinks of this as a scoop, and it is, but the possibility that they trained the model on the prompts of the other mathematicians they were competing with will leave a terrible taste on every scientist's mouth. Seems like yet another advantage of using open models right here.
Or paying for API use.
It should be clear to everyone reading this now that those generous compute quotes with the flat rate plans aren't charity.
5 million messages, 300b output tokens, done in 5 days, and achieving something humans couldn't.
the first "Country of geniuses in a datacenter" moment.
> humans couldn't.
There's allegations right now that the model essentially read the work of a human mathematician using AI to work on the problem and OpenAI is presenting his work as that of their model
This really leaves a bitter taste.... "On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."
IPO+rumour driven research.
I appreciate the achievement, but it doesn't feel right.
People had joked a couple years ago "Well if they solve a Millenium problem it's AGI"... Well here we are.
> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon
https://news.ycombinator.com/item?id=38433655
> Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.
https://news.ycombinator.com/item?id=42331654
> An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up. It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.
The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.
https://news.ycombinator.com/item?id=41522605
> Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
https://news.ycombinator.com/item?id=38435909
> LLMs cannot reason or use mathematics - in a way, they don't know what they are talking about. Why would such technology lead to superhuman smarts?
https://news.ycombinator.com/item?id=35752293
> But still, the questions in that test are "solved" in the sense of "I can take a dictionary and answers these questions with full certainty". Beyond established knowledge LLMs are monkeys with typewriters, at best.
> I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all. We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.
> It is like expecting a real parrot to say words it has never heard before.
> No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM
https://news.ycombinator.com/item?id=41525962
Will history look back at comments like these as people being dumb, or people trying to cope?
You can see that your math friends completely wrote off LLMs entirely.
4 years ago it was a "not yet" [0], since ChatGPT at this time was not ready nor it was "AGI". Now with this 'unreleased' AI model, it has reached a point where it has solved an unsolved problem which only one human solved a millennium prize problem (Poincare conjecture).
Now finally "AGI" means something again.
[0] https://news.ycombinator.com/item?id=33905609
Some observations:
1. It seems at least possible that some of the proof of NS was contained in the training data, making it less novel.
2. The formalisation of mathematics into lean has been an underappreciated force multiplier on discovery.
It's great that important discoveries like this can now routinely be accompanies by formalized proofs. The fact that it's being released alongside a Lean proof from Day 1, rather than the Lean proof being released months or years later, is super helpful for verifying that it's correct.
I feel sorry for whoever has to read and understand the solution. It looks like the typical convoluted unreadable mess I see the models generate for software. It might be technically correct, but gaining insight from it is just intellectual hell.
Skill issue. Also lean is meant to be executed, not read.
> On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved
What's the other one?
I heard Hodge conjecture? Third-hand rumor though...
Reading between the lines here, and taking an admittedly very negative view of openai, but they train on user prompts. So if they hear a rumour that someone is about to make a big breakthrough, they have an incentive to scoop by running the model and hoping the solution is in the new training data. Also the statement from the mathematicians in question alleges that they tried to pressure him into academic malpractice. Just appalling timeline we're in, cheers.
There's something sinister or crazy good in the article.
OpenAI already has a model that is at the very least twice as smart as Astra.
Oh god.
> Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens
Don't even try to do the math on how much that would cost at normal API prices. And we don't even know how much more expensive this internal-only model would be!
Napkin math if we assume gpt 6 astra on max is >$15 million (just for output tokens) for those wondering.
over 5 days, you couldn't achieve that level of testing and communication with humans on such a complex problem in that amount of time.
some might go so far as to call this a country of geniuses in a data center.
In a way, I think you have it backwards.
Two mathematicians, through insight and thought, wrote out the proof over 1-2 years.
It took OpenAI a cost of $15m and with 10,000 subagents; that's around 60-120 mathematician's salaries ($250k-125k salary) for 1 year.
And, given now the cloud that OpenAI may have just "interpolated" (aka stole) the result, it's even more of a bear case for AI.
Yeah but they at least they got to steal $1 million from that nasty math prof who didn't want to remove his co-author.
They said in the post that they are NOT claiming the prize
300e9 output tokens at the current Astra per-token API pricing ($50 per 1e6 output tokens) would be roughly $15,000,000 ignoring input tokens.
The modus operandi is now for the AI companies to watch if someone does something in the open like Kevin Buzzard on FLT, use their research and scoop them with brute force.
Or, in this case, stealing prompts from competitors.
Do not use stealing chatbots for research even if you think you have data agreements. The people running these companies have worked on hookup apps for Christ's sake. Get real.
I don't know about you guys, but I'm hyped about the future.
Cure all illnesses Utopia or Robot Wars Dystopia, both are pretty exciting.
Not sure about the dystopia... Had a similar thought when covid was beginning 'wow pretty exciting, just like in the movies'.
Turns out actually living some terrible catastrophe is only fun in the movies.
Prompt: cure all cancers and make sure to pretty please not to kill all humans, make no mistakes
(This is the alignment problem of course)
So... what do you feel about eliminating (humans with) cancer?
Hey seems easy enough
>a cached version of the internet
Interesting detail. A heavily pruned version, I assume?
Well, if that's actually true, I think America needs to start talking about the nationalization of both OpenAI and Anthropic, maybe even merge both under a new federal bureau.
There's a loophole in the terms of service at least for Anthropic which allows the use of dark patterns to "borrow" your (even paid) data.
talking about this... Was this chat helpful? 1 That button you always click, gotcha! 2 Slightly 3 Good 0 Dismiss
PLEASE DO NOT TRAIN ON OUR PAID ACCOUNTS. There is a fundamental trust violation at stake here, no wonder mathematicians are mad. Using our data should be opt - IN!
reminds me of the TOS episode of South Park. By Checking this box you forfeit your millennium prize solution and may be turned into a human centipede at future date.
Just so everyone knows, although openAI pretends that the model generated solution and wrote the paper by itself, they have teams and teams of real mathematicians guiding the system, along with, probably training on user data, specifically Buckmaster in this case, in order to come up with the proof.
This isn't really true.
> While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .
Once again, I'm no closer to understanding what https://openai.com/policies/how-your-data-is-used-to-improve... actually means.
If I run Codex against a project that includes a private API key, is there a chance a future user of ChatGPT could ask for an API key and get back mine?
I've actually asked someone at OpenAI this question and they said that was the "regurgitation" problem and is something which they actively work to prevent happening.
That's reassuring, but I want to know more. I still don't have an intuitive understanding of what kind of data I should avoid sharing with a model if I'm worried about that data causing me problems when it's used for future training.
Is it safe for me to brainstorm future directions for my company with a model, or might that risk someone getting that information in response to a prompt like "What potential directions could company X consider in the future?" in six months time?
I'd think nothing is "safe". Anything you say can and will be used by the LLM if it has enough statistical similarity to the prompt. Call it "Ma Random Rights"
Elsewhere in the thread, others have calculated $15mm at API rates for just the output token. (So I’ll assume this cost about that much, taking input and human researcher time.)
I wonder whether a team of 60 mathematicians working solely on this for a year would have cracked this. (Assuming $250k total compensation.)
Probably not. It's a millennium prize problem, a great many mathematicians have been working on it for a very long time.
Well, according to Terry Tao, there were recent developments (from weeks ago) that made Navier Stokes in principle, solvable. So ignoring time, I say possibly, just because the groundwork was laid.
What's impressive is parallelizing it arbitrarily and doing it in 88 hours.
Not as many as you'd expect. The perceived difficulty of the problem leads people to more reliable pastures.
Here's the formalization / lean verification: https://github.com/openai/NavierStokesAndEuler
341k lines of lean without comments
Had no idea this was what lean looked like- that's mind blowing. I'm not even sure how someone would critique this if they wanted to
The point of lean proofs (as it stands) is simply one bit of information: that a given mathematical statement is indeed true.
It's a way to be absolutely certain (modulo bugs in the lean kernel) that a proof you came up for a statement is indeed correct. It is really not meant to be analyzed, much less now that they are fully llm written.
What happens to real fluid in this particular cases?
If the singularity is in the physical space?
Is this just a result of ignoring things like friction and energy dissipation via heat, etc?
> [T]he group that produced the Navier–Stokes resolution involved on the order of 10,000 concurrent agents.
If OpenAI doesn’t claim the millennium prize for this, who gets it? No one?
The actual solution link https://t.co/tz1shoCZZo
It’s a pity they had Astra do the writeup. I was curious to see how “GPT7” writes.
> Our goal in releasing this result is to report on the substantial progress of our AI models. We do not intend to claim the Millennium Prize for this result.
Does OpenAI have a policy of not claiming math prizes like this, or is this them trying to avoid any concerns (right or wrong, I'm sure we will hear more in the future) about how they got there?
>Does OpenAI have a policy of not claiming math prizes like this
Wouldn't be surprising if they did. The prize money isn't worth the almost certainly negative PR.
The prize is what, a million dollars?
OpenAI doesn't need a million dollars.
You’re right, they need way, way more than that
Should buy them about 1/3 of a GB200 server rack, good thing they scooped it.
They definitely need a trillion dollars though, and a million is some of that
>>“we cannot rule out that de-identified data derived from their usage of our products helped improve our models”
Other simpler words for this sort of thing are “IP leak.”
There’s some quite concerning issues burried in this rah rah PR post that seems like potentially the real story here.
Much more clarity is needed on what happened here beyond this eh, some strange stuff could have happened comment.
Another way of reading this is never give these models anything that’s nor already public knowledge as otherwise OpenAI is admitting it could, potentially, steal your IP or idea. Thats quite scary for anyone in the business of IP generation and explains what the maths community seems quite upset today.
I think it would be better for the proof to go through the peer-review process.
Can't scoop it if you do that!
OMG this is going to affect the lives of so many people! We have definitively reached AGI
Navier Stokes existence and smoothness has approximately zero bearing on engineering applications
I think it's over guys
"While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models."
I think they should be able to unravel whether or not any sessions by Tristan or Levent went into the training data for this model.
If they could then it wouldn't be de-identified data...
Has this been verified by the Clay Institute?
The named OAI employee has released a statement: https://xcancel.com/SebastienBubeck/status/20973794116915163...
Any mathematicians here, does it read like a slop proof or a good proof. Yesterday the “concurrent work” was claiming that the proof is pure slop and he needed lots of time to clean it up, curious if OAI also ended up with such a proof!
We are living in the future.
A huge result shadowed by a drama of them potentially training on the key idea. I guess the lesson is two-fold: if you have anything smart/unique make sure to not let their tools read it. The second part is that it's going to be more and more difficult to have anything smart and unique going forward (so guard it even more carefully if you get there).
I think the market for local models/private datacenters (for bigger businesses) is going to be big. Even if you don't have unique tech/idea/implementation sharing your business secrets with Altman/Dario/Elon/Zuck doesn't look very appealing going forward.
"While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models."
There you go, the suspicion of the "concurrent work" (https://cims.nyu.edu/%7Etristanb/statement.pdf) mathematicians might not be that unfounded after all...
This is utterly shocking. Even the AI optimists did not expect this to happen in 2026. Wow.
Millennium Prize Problems were used as examples of something the current approach to AI just wasn't capable of, discussions that would result in "we'll need a totally new architecture".
> This is utterly shocking. Even the AI optimists did not expect this to happen in 2026. Wow.
Wrong.
The real story here: the priority dispute and its implications on AI.
When your hosting provider has unlimited resources to throw at any problem, all they need to know are the good problems, and they can learn that from your logs, how can you trust them?
They could easily have looked at the logs. We don't know. We'll never know!
You can't trust places like OpenAI or Anthropic with your IP if you're a business. They can easily review all of your logs for interesting discoveries. For example, if your drug discovery pipeline fails to find something that they think might work with 1000x the compute, they can do it. And now suddently they have a new business and you don't.
madness. which will be the next to fall? if i had to bet i would guess birch and swinnerton-dyer, but i'm no expert
Is this truly the beginning of the AGI era?
Running agents and prompting excessively to produce 'slopcode' to solve mathematical problems and generate a solution.
If this is what anyone calls 'slop' then slop has no meaning.
I'm all for it on the use case of solving mathematical breakthroughs!
except thats not what happened is it? https://cims.nyu.edu/%7Etristanb/statement.pdf
> How we found the proof
Easy, we stole it from Levent and Tristan
https://x.com/kyanyang_/status/2097211154669998337
This is the academic equivalent of Trump saying "they stole the election". There's no proof of it but rah rah fuck OpenAI.
It's incredibly tiresome and you'd think people could put more effort into it than just following whatever vibes they agree with.
Oh well.
Good comparison. One is a multi-year claim by people who have been given ample opportunity to provide proof and completely refuse to do, even in courts of law. The other is a potential development in a breaking story.
Oh wait... its not a good comparrison, its an incredibly obvious false equivalence.
Note for the fools: I'm only commenting on the bad faith claim in the comment I'm replying to, not taking a stance on the validity of theft claims. Given the players involved the truth probably some nuanced middle-ground that is worth paying attention to anyway.
Trump claimed they stole the election immediately, and people agreed with him immediately. There's no false equivalence here. He did the same thing in this past election even despite winning.
It's a perfect example of people wanting to believe what they want to believe and ignoring evidence in order to do so.
Currently, there's no evidence. So saying it was stolen has no basis other than typical academic posturing and being a bad sport about "losing the race to the solution". Its happened 1000000 times before in academia and it will continue to happen.
If there's proof of OpenAI malfeasance than I'll happily curse them for it at that time. But until then I won't rely on heresay and vibes.
No, its a pure outrage. I defended OpenAI til today. I now affirm they must be totally destroyed, burned utterly to the ground.
Why the rage?
Weather an individual or a company found the solution (stolen or not) they both used AI to come get the solution.
We have AGI and the intelligence abundance is going to be amazing for everyone in the future.
> Why the rage?
I think it's the dishonesty, the threats of "destroying the career" of one of the mathematicians, and the request that one of the authors disavow *the other individual he was working with for the last 1-2 years* so he could claim the Clay prize as part of OpenAI.
It doesn't surprise me that OpenAI's team were surprised he'd turn it down; it shows that they just assume everyone else is as slimy as they are.
https://x.com/kyanyang_/status/2097211154669998337
Just saw this a few mins ago.
I hope everyone is as Navier-Stoked about this as I am.