we reached a point where there will always be demand for tokens.
suppose the current frontier models are final and there is no more progress. it still wouldn't matter because the cheaper the tokens the more demand there will be which will drive the silicon demand onward.
consider what you can do if a billion tokens cost cents: you could create a scaffold that would literally burn ungodly numbers of tokens on every conceivable angle. you could create scaffolds that don't just generate the next token but perform MCTS then at the end you keep the best results.
another way to look at it, we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem. but the combinatorics prohibit random exhaustive search. LLM's make exhaustive search somewhat more tractable, because there will probably exist a prompt/chain of thought which can yield it (especially when you can always get experts to poke it with cheap ideas).
creating silicon fabs is probably the most reliable investment at the moment, because there will always be room around the sun. orbital launch is probably less reliable because ocean floors still have a lot of room but it will depend on nuclear regulatory innovations. amusingly, had SpaceX actually gone up it would actually draw parallels to the dotcom bubble as being too soon.
note that tokens aren't just text anymore, MCP is very effective. if nothing else with billions of cheap tokens you will be able to exhaustively create endless virtual worlds with just Unreal Engine MCP - this is the absolute floor. entertainment without end.
and this all with just the current generation models frozen in time. but suppose only the intelligence is frozen, suppose training still works. if it's cheap enough you can always just dump compute on RL on the MCP directly. and the MCP can be something like silicon design...
There is no evidence that billions of tokens will ever cost cents. The raw infrastructure cost alone is massive. That's why you see services now charging usage based pricing, because continuing to subsidize inference costs is unsustainable long term.
cents per billion is only 3 orders of magnitude reduction in cost. I can see 6 or so happening in the next decade or so.
1. Like any commodity, I expect the price to converge to the marginal cost. The marginal cost is the price of electricity which is currently 10% or so of the cost of inference.
2. I expect the cost of electricity to go down an order of magnitude. Solar power and batteries are getting cheaper fast.
3. As the price of electricity starts dominating costs, you can drastically reduce costs by by taking advantage of electricity price variations. Build data centers all around the world and answer requests from the part of the world where the sun is shining.
4. Moore's law
5. Algorithm improvements
6. Moving from general purpose chips to chips that are highly optimized for inference.
The main trend is that models are moving out of cloud to local setup to avoid paying for tokens. This becomes possible due to free open-source models like Kimi K3 that already reached level of the best commercial models.
The second trend is that AI is not mature enough to be used as a human replacement. Maybe as a helper, but even then they need close guidance and check. All these talks about autonomous agents, AI employees. etc - this is all hype that current level of AI technology cannot deliver. Realization of this fact will cause bubble deflation.
> The main trend is that models are moving out of cloud to local setup to avoid paying for tokens. This becomes possible due to free open-source models like Kimi K3 that already reached level of the best commercial models.
Who on earth is running a 1.8T parameter model locally? The hardware to do that with a usable token/s rate for a single person is (at least) well into 5 figure territory.
Everyone feels it's a bubble, but the key question is when it will burst. People keep pointing this out in so many places, from Closed loops to everything else. I wish someone would just predict when that moment will come
I've seen so many of these articles that I'm actually looking forward to a YouTube video or article that talks about a techno-feudal future, where API tenant farmers become the scenario if AI companies succeed. Sometimes I just want to see something new.
There is a big chance it is happening now. SpaceX lost almost 50% of its ATH value. It will continue dropping. OpenAI and Anthropic won't be able to raise money but they need to. Things are about to get uglier fast.
I also think this bubble will burst. But what I'm curious about is this: most of these arguments are just unfalsifiable alibis. They're just claims, and I want to know when and how it will burst. Most scenarios are way too vague
I'm not going to predict the moment, because I thought the dotcom bubble would pop back in 1999. I could see how overvalued things were, what I couldn't predict was how long the market would remain irrational.
For me, my blind spot has been China. They've maintained their bubble economy for decades longer than I (and a few other smarter people than me) think possible.
Honestly, I'd call that more of a wish than a prediction. Because most predictions specify which quarter something will burst. Saying that an industry structure will turn out a certain way is just stating the obvious, since everything eventually declines. Every new industry grows fast in the early days and then bursts. That's just wishful thinking.
Claims like 'it will collapse someday' are actually unfalsifiable. Predictions that specify a quarter, on the other hand, are not. This is exactly why I think so-called pundits are useless. They repeat unfalsifiable claims over and over, and when it finally happens later, they say 'I told you so.' How ridiculous is that? I can make that kind of claim too. 'The US will collapse.' Sure, it will. But who knows if it'll even exist in a thousand years? So honestly, if there's no timeframe, it's meaningless.
But most of what gets posted on HN is exactly this kind of unfalsifiable punditry. I'd rather write a new scenario. A scenario where AI succeeds, we're all classified as API tenant farmers, only a few influencers chosen by OpenAI and Anthropic survive, most programming companies shrink, and a new mega-tech emerges. That would be more fun to write.
We will see if the outcomes are similar - dotcom sucked and hurt a lot of individuals and small businesses. This is different and the effects may be a lot worse.
Around 2002 there was way more internet bandwidth and infrastructure available than utilized, eventually (not immediately) leading to business models like Netflix current one. I do think a lot about what the AI analogy will be.
If you can wait until one of the major companies goes bankrupt (assuming that that happens, of course) before buying a new PC, then it should be a lot cheaper than today. Because when a company goes bankrupt, a lot (or possibly all depending on how bankrupt they are) the hardware they bought will be sold off to pay their debts, meaning a LOT of now-scare RAM will go back on the market, bringing prices back down as supply suddenly catches up with demand.
It's not guaranteed, and the pent-up demand for RAM from all the other people who had been holding off on buying new hardware will mean that supply won't exceed demand for quite some time. But that's my strategy: wait for one of the companies who are currently outspending their resources to go bankrupt, then buy hardware once the supply is no longer quite so ridiculously tight as it is now.
we reached a point where there will always be demand for tokens.
suppose the current frontier models are final and there is no more progress. it still wouldn't matter because the cheaper the tokens the more demand there will be which will drive the silicon demand onward.
consider what you can do if a billion tokens cost cents: you could create a scaffold that would literally burn ungodly numbers of tokens on every conceivable angle. you could create scaffolds that don't just generate the next token but perform MCTS then at the end you keep the best results.
another way to look at it, we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem. but the combinatorics prohibit random exhaustive search. LLM's make exhaustive search somewhat more tractable, because there will probably exist a prompt/chain of thought which can yield it (especially when you can always get experts to poke it with cheap ideas).
creating silicon fabs is probably the most reliable investment at the moment, because there will always be room around the sun. orbital launch is probably less reliable because ocean floors still have a lot of room but it will depend on nuclear regulatory innovations. amusingly, had SpaceX actually gone up it would actually draw parallels to the dotcom bubble as being too soon.
note that tokens aren't just text anymore, MCP is very effective. if nothing else with billions of cheap tokens you will be able to exhaustively create endless virtual worlds with just Unreal Engine MCP - this is the absolute floor. entertainment without end.
and this all with just the current generation models frozen in time. but suppose only the intelligence is frozen, suppose training still works. if it's cheap enough you can always just dump compute on RL on the MCP directly. and the MCP can be something like silicon design...
There is no evidence that billions of tokens will ever cost cents. The raw infrastructure cost alone is massive. That's why you see services now charging usage based pricing, because continuing to subsidize inference costs is unsustainable long term.
cents per billion is only 3 orders of magnitude reduction in cost. I can see 6 or so happening in the next decade or so.
1. Like any commodity, I expect the price to converge to the marginal cost. The marginal cost is the price of electricity which is currently 10% or so of the cost of inference.
2. I expect the cost of electricity to go down an order of magnitude. Solar power and batteries are getting cheaper fast.
3. As the price of electricity starts dominating costs, you can drastically reduce costs by by taking advantage of electricity price variations. Build data centers all around the world and answer requests from the part of the world where the sun is shining.
4. Moore's law
5. Algorithm improvements
6. Moving from general purpose chips to chips that are highly optimized for inference.
> we know that there is a document in the space of all possible documents which holds an answer to every conceivable solvable problem.
So you think reality is static and not an ever growing ever changing process?
>we reached a point where there will always be demand for tokens.
yes but who will pay for them.
I fear it will be tax payers and consumers of high end products... everyone else else can go eat dirt.
How is this related to the post?
You are saying that AI is useful.
That doesn't mean there isn't a bubble. The internet is useful and we still had a dot com crash.
vibecoding is as much demand for AI tokens as pets.com was for early internet ecommerce. Kimi K3 is already crashing this.
The main trend is that models are moving out of cloud to local setup to avoid paying for tokens. This becomes possible due to free open-source models like Kimi K3 that already reached level of the best commercial models.
The second trend is that AI is not mature enough to be used as a human replacement. Maybe as a helper, but even then they need close guidance and check. All these talks about autonomous agents, AI employees. etc - this is all hype that current level of AI technology cannot deliver. Realization of this fact will cause bubble deflation.
> The main trend is that models are moving out of cloud to local setup to avoid paying for tokens. This becomes possible due to free open-source models like Kimi K3 that already reached level of the best commercial models.
Who on earth is running a 1.8T parameter model locally? The hardware to do that with a usable token/s rate for a single person is (at least) well into 5 figure territory.
Everyone feels it's a bubble, but the key question is when it will burst. People keep pointing this out in so many places, from Closed loops to everything else. I wish someone would just predict when that moment will come
I've seen so many of these articles that I'm actually looking forward to a YouTube video or article that talks about a techno-feudal future, where API tenant farmers become the scenario if AI companies succeed. Sometimes I just want to see something new.
There is a big chance it is happening now. SpaceX lost almost 50% of its ATH value. It will continue dropping. OpenAI and Anthropic won't be able to raise money but they need to. Things are about to get uglier fast.
I also think this bubble will burst. But what I'm curious about is this: most of these arguments are just unfalsifiable alibis. They're just claims, and I want to know when and how it will burst. Most scenarios are way too vague
isn't that the point of bubbles: nearly impossible to predict when they'll burst?
I'm not going to predict the moment, because I thought the dotcom bubble would pop back in 1999. I could see how overvalued things were, what I couldn't predict was how long the market would remain irrational.
For me, my blind spot has been China. They've maintained their bubble economy for decades longer than I (and a few other smarter people than me) think possible.
Honestly, I'd call that more of a wish than a prediction. Because most predictions specify which quarter something will burst. Saying that an industry structure will turn out a certain way is just stating the obvious, since everything eventually declines. Every new industry grows fast in the early days and then bursts. That's just wishful thinking.
Claims like 'it will collapse someday' are actually unfalsifiable. Predictions that specify a quarter, on the other hand, are not. This is exactly why I think so-called pundits are useless. They repeat unfalsifiable claims over and over, and when it finally happens later, they say 'I told you so.' How ridiculous is that? I can make that kind of claim too. 'The US will collapse.' Sure, it will. But who knows if it'll even exist in a thousand years? So honestly, if there's no timeframe, it's meaningless.
But most of what gets posted on HN is exactly this kind of unfalsifiable punditry. I'd rather write a new scenario. A scenario where AI succeeds, we're all classified as API tenant farmers, only a few influencers chosen by OpenAI and Anthropic survive, most programming companies shrink, and a new mega-tech emerges. That would be more fun to write.
What is this Kurzgesagt knockoff bullshit?
Everyone has an opinion, but what is their rate of successful predictions?
We will see if the outcomes are similar - dotcom sucked and hurt a lot of individuals and small businesses. This is different and the effects may be a lot worse.
At least the dot com crash left us with m useful infrastructure. And it didn’t price regular customer out of the computer market.
Going to have to sell a kidney to build a new gaming PC the next few years.
Around 2002 there was way more internet bandwidth and infrastructure available than utilized, eventually (not immediately) leading to business models like Netflix current one. I do think a lot about what the AI analogy will be.
If you can wait until one of the major companies goes bankrupt (assuming that that happens, of course) before buying a new PC, then it should be a lot cheaper than today. Because when a company goes bankrupt, a lot (or possibly all depending on how bankrupt they are) the hardware they bought will be sold off to pay their debts, meaning a LOT of now-scare RAM will go back on the market, bringing prices back down as supply suddenly catches up with demand.
It's not guaranteed, and the pent-up demand for RAM from all the other people who had been holding off on buying new hardware will mean that supply won't exceed demand for quite some time. But that's my strategy: wait for one of the companies who are currently outspending their resources to go bankrupt, then buy hardware once the supply is no longer quite so ridiculously tight as it is now.