I'm also grateful to the Chinese labs for providing workarounds for the walled gardens that the US based AI companies are attempting to create.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
There are 1000 Chinese labs. They are involuting, they cannot coordinate and the state won't let them coordinate because the state wants domination, not actual profits for anybody. So the market forces are allowed to dominate.
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export.
If you want an essay about this from a VC, read this one
They want the Western labs to keep digging themselves into a hole? You can do so by egging on the true believers on your side to egg on the labs on the other side.
As far as I can tell, neither of the frontier US labs have referred to distillation as "stealing", but someone please provide a link if I'm wrong.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
The first sentence is: "If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse."
Since I have no idea what "spawn-camped" means I gave up reading the rest.
I'm also grateful to the Chinese labs for providing workarounds for the walled gardens that the US based AI companies are attempting to create.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
Performance optimizations don't just help the western labs, they also help with running more powerful/useful LLMs locally.
If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.
> people will soon *stop paying
Think you missed a word there.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Any ideas?
There are 1000 Chinese labs. They are involuting, they cannot coordinate and the state won't let them coordinate because the state wants domination, not actual profits for anybody. So the market forces are allowed to dominate.
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export.
If you want an essay about this from a VC, read this one
https://earnedintuition.substack.com/p/involution-without-ex...
Occam's razor: Going to closed-source just to hide KV-cache optimizations seems silly?
They want the Western labs to keep digging themselves into a hole? You can do so by egging on the true believers on your side to egg on the labs on the other side.
Market manipulation or slowing down demand for chips for a bit/moving the offer elsewhere temporarily?
As far as I can tell, neither of the frontier US labs have referred to distillation as "stealing", but someone please provide a link if I'm wrong.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
OpenAI is jobbing (in professional wresting terminology) hard right now.
Appropriate domain name
Just assumptions, nothing backing it. So maybe I'd sit out calling others out.
Edit: Apt domain.
How is it just assumptions? They provide the data to back up their claims.
The first sentence is: "If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse."
Since I have no idea what "spawn-camped" means I gave up reading the rest.
It's from video games, where a player "camps" near the spawn point and kills newly-spawned players, presumably with better equipment.
It's not that niche, if you've been online a little bit you'd know this expression.
It's gamer terminology.
In a PvP (player vs player) game, if you kill a player the moment they spawn into the game arena, that's called "spawn-camping".