Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how and open source inference engine undermines them.
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
Today. As new suppy comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
DeepSeek is a fixture among privacy-conscious local LLM advocates, but they are still battling for marketshare.
This is advertisement for them, and more so for large entities (like governments and corporations) looking for digital sovereignity: the more they are reassured they can run of their stack locally when needed, the more they will use the cloud offer too.
I’m pretty sure this is rage bait. The idea that China is more interested in virtue signaling than pulling off a power move that will improve their chess position is not worth a serious reply.
as opposed to the current crop being backed by the US state? The ones that are basically crying foul at competition? The ones that unilaterally tries to dictate based on political friendliness and compliance (ala, anthropic getting banned)?
At least with open models, you can own the hardware and do the end to end. The current bottleneck is hardware availability at low cost, but i am guessing the chinese state is looking to build their own chip supply chains. I suspect they will succeed.
I believe majority of the AI inference (and probably training) infrastructure runs on Linux, so I'd certainly consider it a major player in the AI race.
I'm wondering how it benefits Deepseek. Why would they willingly give up their trade secrets?
It's obviously very valuable, as they are running a profitable company. It follows the same trend as when they released their cache optimizations a few months back: why do it? What followed their release of this technology was that Anthropic and OpenAI silently released models which were on-par or similarly intelligent, but "surprisingly" had much cheaper cached tokens. So their inventions were obviously just worked into their competitors, who just got much greater margins + became more profitable.
The same thing goes here: I'm sure there are inventions and optimizations which are unknown to Deepseeks competitors which yet again will be worked in. So once again, they're letting their trade secrets go.
My question is: why? How does Deepseek benefit from this? There must be some gain I'm not seeing as they keep doing this play over and over, and I doubt it's "for the good of open source", as China is practically as isolated as you can get. To believe that this is in the spirit of the open source community is heavily naive.
That is what I am not sure about...It is like, is having a public key closer to hacking the account than not having it? Theoretically, it is, but practically, it is not...
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
That's strategic sabotage to say the least.
Ban all open weight models! Cried the VCs.
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how and open source inference engine undermines them.
What moat?
It's the USA, you can just ban it.
https://public-inspection.federalregister.gov/2026-21447.pdf
Thats like sticking your head in the sand :)
That’s a bingo
A nuclear bomb
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
Source: https://zhuanlan.zhihu.com/p/2095560295930508953
For the ones who are still wondering how so much profits, inference is a highly profitable domain: https://inferencex.semianalysis.com/profit-estimator
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
Accounting for initial acquisition cost and typical depreciation/failure rate? I assume they already consider colo price, electricity, etc
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
Today. As new suppy comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
$0.00051192 per M/tok ?
Assuming a 64 card cluster
Notably missing their kv cache system
It's exceptionally optimized... 890 bytes per token. At standard 32K context, that's less than 30MB.
1. https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
2. https://www.mindstudio.ai/blog/deepseek-v4-1-flash-specs-arc...
https://x.com/lightseekorg/status/2112470406158561730
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Data and GPU compute capacity.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
How to destroy the entire US economy without firing a single bullet
Not written in Rust? How dare you?
Has to be something other than Rust to be able to be "rewritten in Rust"
Their inference server/service (wrapper around this engine) could still be written in Rust, this is just the engine.
This is written in Rust: https://github.com/deepseek-ai/deepseek-recipe
This DeepSeek Recipe is also mentioned here: https://zhuanlan.zhihu.com/p/2095560295930508953
Given how much the models like React and JavaScript I guess we should be thankful its Python.
Apart from virtue signaling, why do this? I can't see how this benefits Deepseek
DeepSeek is a fixture among privacy-conscious local LLM advocates, but they are still battling for marketshare.
This is advertisement for them, and more so for large entities (like governments and corporations) looking for digital sovereignity: the more they are reassured they can run of their stack locally when needed, the more they will use the cloud offer too.
I’m pretty sure this is rage bait. The idea that China is more interested in virtue signaling than pulling off a power move that will improve their chess position is not worth a serious reply.
I'm curious, are projects like Linux "virtue signaling" to you?
No, and it's also not a major player in the AI race, backed by the Chinese state
> backed by the Chinese state
as opposed to the current crop being backed by the US state? The ones that are basically crying foul at competition? The ones that unilaterally tries to dictate based on political friendliness and compliance (ala, anthropic getting banned)?
At least with open models, you can own the hardware and do the end to end. The current bottleneck is hardware availability at low cost, but i am guessing the chinese state is looking to build their own chip supply chains. I suspect they will succeed.
I believe majority of the AI inference (and probably training) infrastructure runs on Linux, so I'd certainly consider it a major player in the AI race.
Perhaps this is evidence there's not as much Fu Manchu moustache twirling scheming as it seemed?
It's releasing open source code. This is a good thing for the world, even if it impacts the portfolio of US VCs.
I'm wondering how it benefits Deepseek. Why would they willingly give up their trade secrets?
It's obviously very valuable, as they are running a profitable company. It follows the same trend as when they released their cache optimizations a few months back: why do it? What followed their release of this technology was that Anthropic and OpenAI silently released models which were on-par or similarly intelligent, but "surprisingly" had much cheaper cached tokens. So their inventions were obviously just worked into their competitors, who just got much greater margins + became more profitable.
The same thing goes here: I'm sure there are inventions and optimizations which are unknown to Deepseeks competitors which yet again will be worked in. So once again, they're letting their trade secrets go.
My question is: why? How does Deepseek benefit from this? There must be some gain I'm not seeing as they keep doing this play over and over, and I doubt it's "for the good of open source", as China is practically as isolated as you can get. To believe that this is in the spirit of the open source community is heavily naive.
> I'm wondering how it benefits Deepseek. Why would they willingly give up their trade secrets?
Someone already suggested it may be same as with other open source software, but you rejected that comparison with an ad absurdum.
Why make open weights models at all, by your reasoning? Unlike the Anthropics of this world, DeepSeek isn't hell-bound on world domination
I am wondering how much stuff (in terms of malicious behavior) one can hide in the (open) weights?
Certainly a lot less compared to closed weights :)
That is what I am not sure about...It is like, is having a public key closer to hacking the account than not having it? Theoretically, it is, but practically, it is not...
deepseek is a pawn. china cant compete and can't use our hardware so they'll burn it all down. the cold war continues.
Seems like they're competing pretty damn well by just about every metric.