I just did a cost breakdown of owning a GPU vs paying for tokens.
An RTX 5060 Ti 16GB card costs about $800 on Amazon right now. Running 18.4 hours a day ( off peak electric rate schedule ) it can process approximately 109.3 million tokens per month. The card has a 90 day return window. Credit card interest + electricity puts the cost at $80 total out of pocket, to use this card for 3 months and return it.
Paying an AI provider came out to $1,639.50 for the same 3 months of 18.4 hour a day usage.
You know its a bubble when ... $80 vs $1,639.50
I understand that if you're a major corporation, or a researcher that needs 100+ gpus for a large project, unless you have access to your own supercomputer, the economics will be different.
This was for "individual coder at home" scenarios.
I just did a cost breakdown of owning a GPU vs paying for tokens.
An RTX 5060 Ti 16GB card costs about $800 on Amazon right now. Running 18.4 hours a day ( off peak electric rate schedule ) it can process approximately 109.3 million tokens per month. The card has a 90 day return window. Credit card interest + electricity puts the cost at $80 total out of pocket, to use this card for 3 months and return it.
Paying an AI provider came out to $1,639.50 for the same 3 months of 18.4 hour a day usage.
You know its a bubble when ... $80 vs $1,639.50
I understand that if you're a major corporation, or a researcher that needs 100+ gpus for a large project, unless you have access to your own supercomputer, the economics will be different.
This was for "individual coder at home" scenarios.
That card will only run a tiny model though, no?
To be fair we don't know which direction that arbitrage will collapse to.