I think it's the memory which is the bottleneck more so than processing speed and the obvious levers to push on are:
- more memory efficient models
- a whole system approach to getting better performance out of a less capable model
- more memory
the memory crisis, Micron shutting down a beloved brand that built trust over almost 30 years, and all of that is an economic externalization of memory as the bottleneck.
Do you have evidence for this claim? The most-advanced AI we've seen deployed by the US federal government (eg. NRO Sentient) does not seem very far ahead of the SOTA whatsoever.
I think it's the memory which is the bottleneck more so than processing speed and the obvious levers to push on are:
- more memory efficient models
- a whole system approach to getting better performance out of a less capable model
- more memory
the memory crisis, Micron shutting down a beloved brand that built trust over almost 30 years, and all of that is an economic externalization of memory as the bottleneck.
The government is using hardware changes to make AI more advanced more cheap, etc... but wait few decades before it gets public and customer friendly
Do you have evidence for this claim? The most-advanced AI we've seen deployed by the US federal government (eg. NRO Sentient) does not seem very far ahead of the SOTA whatsoever.