This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money, which would give them less money, which would slow progress on AI. It assumed all labs in the entire world agree to self-destruct this way and no new labs every start up to continue the work.
No, really:
> As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.
Which doesn’t even follow. It depends on every lab in the world agreeing to self-destruct their valuations and stop competing. That’s a much more impossible ask than anything Dario is asking for.
It also ignores the possibility of new labs starting up, using the open weights, and continuing exactly where the old labs left off.
I feel like I just read the ramblings of someone who got so excited about the headline that they didn’t take the time to think if their argument was sound.
> This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money
That argument is straight from Dean Ball on Twitter. Open-weight models are "decelerationist", which is bad if you're completely AGI-pilled but is actually great if you're either worried about AI Safety falling to an arms-race at the frontier (this is Dario's argument, and note that open weight models are far behind the proprietary frontier, especially since they target widespread lean deployment so they must skimp on total parameters and compute requirements) or Yann-LeCun-pilled (i.e. skeptical about AGI/ASI but optimistic about the practical usefulness of current AIs) like most people in China (including, reportedly, Chinese leadership).
It is a bit incoherent, but it's also... provably compliant in a scorched-earth kinda way?
What are the alternatives?
A regulator saying no? Easy, your headquarters has just moved the Cayman Islands, Ireland, or Switzerland... the US office is just a subsidiary leasing the brand IP and doing marketing.
Or, you just ignore the regulator behind closed doors because you're part of some black budget. You wouldn't be able to talk about that closet back there even if it did exist.
Or, you don't do any of these complicated loopholes and you simply move all the training and inference to a different jurisdiction.
So yeah, the argument is incoherent, but it does have a kind of "only way we can make sure everyone stops playing the game is to destroy the game" mad logic to it.
It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
It's Dario's plan that depends on every lab in the world agreeing.
Step #2 of his essay is "democratic coordination" among frontier labs and step three is coordination with China. I'm suggesting one simple law Congress could pass and every US lab would be covered the same day, whether they like it or not.
Frontier progress is compute bound. Compute is bought with investor money. Investors put up tens of billions because they expect to sell the proprietary model that is created.
You're welcome to think the effect would be smaller than I do. But "it requires everyone to agree" is backwards.
> It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
Laws don’t extend to the entire world. There are AI labs outside of the US that are not that far behind. Forcing US companies to open their weights would hand those labs another advantage, moving the frontier AI to a location outside of any US regulation.
Your proposal doesn’t make any sense.
> The plan that depends on every lab in the world agreeing is Dario's.
I don’t understand how you think creating a law in the US to destroy our AI companies and give an advantage to competitors we can’t regulate accomplishes anything in line with the AI safety debate.
Laws don't extend to the whole world, but markets do. The law should apply to anyone selling model access to Americans, the same way GDPR reaches US companies selling to Europeans.
Frontier development won't simply move abroad, because the GPUs, most of the people, and almost all of the funding is here.
Our European and Asian allies would probably go along with this idea, because it costs us far more than it costs them.
It follows if the goal is to deflate the bubble without popping it, and if the reality under the hood supports the absurd investment. The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
> It follows if the goal is to deflate the bubble without popping it,
The conversation was about AI safety, monitoring, and regulations, not bubbles.
Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
> The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
What makes you think the companies outside of the US wouldn’t just thank the US companies for their open weights and then continue on without any of the regulations but all of the advantages?
> Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
Or maybe you understand less than you think?
Frontier progress is limited by compute, and compute is bought with investor money. Investors fund labs at hundreds of billions because they expect exclusive ownership of the model weights.
If the weights are public the day a model ships, the valuations drop, the funding rounds shrink, and the next training run is smaller or later. Make sense?
The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business. They can cut it off any time. Public access to their models (at subsidized rates) has caused so much indirect harm and financial cost to them that they're very well justified to stop providing access.
Anthropic has already shown they're very comfortable with holding models from the public. Mythos has been around for what, six months, and there's never been a public release. Only the select few blessed by Anthropic have access.
Forcing transparency on public models will just accelerate privatizing access to these models. You don't want to live in that world.
I refuse to believe this is not some sort of marketing piece for author's company.
This is such a short-sighted take and author seems unaware of what damage can be done with powerful open weights models (intentional or not).
Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case). Now imagine state sponsored bio lab leaking strange viruses on yearly basis. We've seen enough evidence already, it is difficult for me to imagine an engineer living at the centre of the change is writing this piece.
It's a different question if you ask me if I trust a single company to police the technology. But with the current state of things, I unwillingly admit that no state entity can do a better job at the moment.
CEOs are motivated by money and power. They will say anything to get either. If they truly wanted to slow down research, they would spend lobby money on the candidates who could enact laws.
Instead, they are partnering with some of the most disliked companies (x, meta) to further their goals.
This is a dumb argument. There are truly a few options, and I see at least two which are not discussed in the original post: banning the publishing of strong open weight models internationally, and nationalizing LLM companies.
I’m not saying I’m for these solution, I’m probably against, but if we are being serious why are we not discussing these options?
1) there would be no incentive to develop a new model (non-distilled) if it had to be released as open weights.
2) frontier models are way more dangerous if they are open. It’s opening Pandora’s box, there’s no going back once they are released if they are too dangerous.
Completely delusional. Step #1 is asking them to give away their product for free and stop raising money. Might as well ask for them to pay their customers since we are just saying stupid shit.
> You seem like a good and principled person, and you have a record of giving things up for what you believe.
i mean, 3 months ago i started to use AI for a project i'm working for around 6 years, full-time but calling a billionaire good and principled is a stretch and when you consider what made him rich, you sound like a 3 year-old kid trying persuade a mom or a dad to get lollipops when your are diabetic
LLMs it's already a economical treat for various fields/people, which not only funnels powers to the elite even more, it does through by violating copyright... everyone who uses this technology is dirt (that includes me and my little dreams of hopefully creating a company which employs minorities on the tech field) but the amount of greed from people who own these servers/GPUs is beyond this world - the world isn't GPLv3 licensed yet, an ethical society wouldn't find excuses to provide and sell LLMs to the masses
An open letter to Dario: if you mean it, give me a billion dollars!
Every time you, or any AI company releases a new frontier model, you have to give me a billion dollars. No strings attached. This will solve the danger of AI. Somehow.
This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money, which would give them less money, which would slow progress on AI. It assumed all labs in the entire world agree to self-destruct this way and no new labs every start up to continue the work.
No, really:
> As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.
Which doesn’t even follow. It depends on every lab in the world agreeing to self-destruct their valuations and stop competing. That’s a much more impossible ask than anything Dario is asking for.
It also ignores the possibility of new labs starting up, using the open weights, and continuing exactly where the old labs left off.
I feel like I just read the ramblings of someone who got so excited about the headline that they didn’t take the time to think if their argument was sound.
> This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money
That argument is straight from Dean Ball on Twitter. Open-weight models are "decelerationist", which is bad if you're completely AGI-pilled but is actually great if you're either worried about AI Safety falling to an arms-race at the frontier (this is Dario's argument, and note that open weight models are far behind the proprietary frontier, especially since they target widespread lean deployment so they must skimp on total parameters and compute requirements) or Yann-LeCun-pilled (i.e. skeptical about AGI/ASI but optimistic about the practical usefulness of current AIs) like most people in China (including, reportedly, Chinese leadership).
It is a bit incoherent, but it's also... provably compliant in a scorched-earth kinda way?
What are the alternatives?
A regulator saying no? Easy, your headquarters has just moved the Cayman Islands, Ireland, or Switzerland... the US office is just a subsidiary leasing the brand IP and doing marketing.
Or, you just ignore the regulator behind closed doors because you're part of some black budget. You wouldn't be able to talk about that closet back there even if it did exist.
Or, you don't do any of these complicated loopholes and you simply move all the training and inference to a different jurisdiction.
So yeah, the argument is incoherent, but it does have a kind of "only way we can make sure everyone stops playing the game is to destroy the game" mad logic to it.
Author here.
It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
It's Dario's plan that depends on every lab in the world agreeing.
Step #2 of his essay is "democratic coordination" among frontier labs and step three is coordination with China. I'm suggesting one simple law Congress could pass and every US lab would be covered the same day, whether they like it or not.
Frontier progress is compute bound. Compute is bought with investor money. Investors put up tens of billions because they expect to sell the proprietary model that is created.
You're welcome to think the effect would be smaller than I do. But "it requires everyone to agree" is backwards.
> It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.
Laws don’t extend to the entire world. There are AI labs outside of the US that are not that far behind. Forcing US companies to open their weights would hand those labs another advantage, moving the frontier AI to a location outside of any US regulation.
Your proposal doesn’t make any sense.
> The plan that depends on every lab in the world agreeing is Dario's.
I don’t understand how you think creating a law in the US to destroy our AI companies and give an advantage to competitors we can’t regulate accomplishes anything in line with the AI safety debate.
Laws don't extend to the whole world, but markets do. The law should apply to anyone selling model access to Americans, the same way GDPR reaches US companies selling to Europeans.
Frontier development won't simply move abroad, because the GPUs, most of the people, and almost all of the funding is here.
Our European and Asian allies would probably go along with this idea, because it costs us far more than it costs them.
> It's a law, so nobody has to agree to anything.
Step 1, form a world government?
Won’t the companies just change what they are selling from access to the model to “compute” and then not technically be breaking the law?
If you're proposing a law that would not be in Dario's best interest, why address the post to him?
I explained this in the post. He seems like a genuinely good person who might actually do something this great.
Dude this site is so funny at times.
Incredible.
Yes, why would you ever address someone with something that is not in their best interest. Especially people that have power.
I agree. We should never do that and let them just live in peace and do their thing.
> It depends on every lab in the world agreeing to self-destruct their valuations and stop competing.
The Chinese labs are already "self-destructing" voluntarily by releasing their weights. Or do they have an actual business model?
It follows if the goal is to deflate the bubble without popping it, and if the reality under the hood supports the absurd investment. The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
> It follows if the goal is to deflate the bubble without popping it,
The conversation was about AI safety, monitoring, and regulations, not bubbles.
Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
> The companies themselves wouldn't do it, but they could be required to if they want to operate in the US.
What makes you think the companies outside of the US wouldn’t just thank the US companies for their open weights and then continue on without any of the regulations but all of the advantages?
> Forcing the weights to be opened to the world accomplishes the opposite of all of that. The argument is incoherent. Or this author doesn’t understand the topic at all?
Or maybe you understand less than you think?
Frontier progress is limited by compute, and compute is bought with investor money. Investors fund labs at hundreds of billions because they expect exclusive ownership of the model weights.
If the weights are public the day a model ships, the valuations drop, the funding rounds shrink, and the next training run is smaller or later. Make sense?
Your whole argument seems to rest on the idea that the AI race is entirely contained within the United States, where we can regulate them.
And that no other country might just take the open weights and continue right on with the advancements in private?
The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business. They can cut it off any time. Public access to their models (at subsidized rates) has caused so much indirect harm and financial cost to them that they're very well justified to stop providing access.
Anthropic has already shown they're very comfortable with holding models from the public. Mythos has been around for what, six months, and there's never been a public release. Only the select few blessed by Anthropic have access.
Forcing transparency on public models will just accelerate privatizing access to these models. You don't want to live in that world.
I refuse to believe this is not some sort of marketing piece for author's company.
This is such a short-sighted take and author seems unaware of what damage can be done with powerful open weights models (intentional or not).
Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case). Now imagine state sponsored bio lab leaking strange viruses on yearly basis. We've seen enough evidence already, it is difficult for me to imagine an engineer living at the centre of the change is writing this piece.
It's a different question if you ask me if I trust a single company to police the technology. But with the current state of things, I unwillingly admit that no state entity can do a better job at the moment.
Dario has been pretty clear and consistent that:
1. AI model safety is the most important thing.
2. open weights decreases model safety
So it seems unlikely that this suggestion would be well-received
CEOs are motivated by money and power. They will say anything to get either. If they truly wanted to slow down research, they would spend lobby money on the candidates who could enact laws.
Instead, they are partnering with some of the most disliked companies (x, meta) to further their goals.
Anthropic supported Alex bores who was one of the most AI safety pilled politicians https://www.theguardian.com/us-news/2026/jun/24/big-tech-new...
I am very skeptical of those 3 faces of American AI labs: Dario, Sam and Elon
Are they trying to stop everyone else from catching up with them or have they hit some kind of roadblock to improve models even further?
But IMO, they're not trying to help humanity
This is a dumb argument. There are truly a few options, and I see at least two which are not discussed in the original post: banning the publishing of strong open weight models internationally, and nationalizing LLM companies.
I’m not saying I’m for these solution, I’m probably against, but if we are being serious why are we not discussing these options?
He doesn't mean it.
He is the CEO of a venture-backed company.
What he says is merely intended to ultimately increase company valuation. Nothing more.
How about reading AGENTS.md...
https://github.com/anthropics/claude-code/issues/6235
1) there would be no incentive to develop a new model (non-distilled) if it had to be released as open weights.
2) frontier models are way more dangerous if they are open. It’s opening Pandora’s box, there’s no going back once they are released if they are too dangerous.
Completely delusional. Step #1 is asking them to give away their product for free and stop raising money. Might as well ask for them to pay their customers since we are just saying stupid shit.
> You seem like a good and principled person, and you have a record of giving things up for what you believe.
i mean, 3 months ago i started to use AI for a project i'm working for around 6 years, full-time but calling a billionaire good and principled is a stretch and when you consider what made him rich, you sound like a 3 year-old kid trying persuade a mom or a dad to get lollipops when your are diabetic
LLMs it's already a economical treat for various fields/people, which not only funnels powers to the elite even more, it does through by violating copyright... everyone who uses this technology is dirt (that includes me and my little dreams of hopefully creating a company which employs minorities on the tech field) but the amount of greed from people who own these servers/GPUs is beyond this world - the world isn't GPLv3 licensed yet, an ethical society wouldn't find excuses to provide and sell LLMs to the masses
The concept is stupid and the rule as stated is stupid.
"Any AI model a company offers to the public has to be released as open weights."
OK great, that means no models released to the public, complete concentration of power.
if they do not release any model to the public: how would they make money then?
Their entire funding model depends on that they generate much revenue _soon_
I feel at this stage it has become a west vs east race of domination. I don’t see them slowing down in the foreseeable future.
Can we all play this game?
An open letter to Dario: if you mean it, give me a billion dollars!
Every time you, or any AI company releases a new frontier model, you have to give me a billion dollars. No strings attached. This will solve the danger of AI. Somehow.
Anthropic will never open the weights, just like how no frontier lab will open up their training data.
What a dumb ask