Inference sounds cooler and more mystical. Any old company can host something, "running inference" requires 10x rock star engineers and a CEO that acts like a badass who wears a leather jacket and has Thoughts about demographics.
That's interesting, the idea that hosting implies a front-end (whether UI or API).
So it's similar to why we don't call Heroku or Vercel "hosting" or "compute" because it's more of a service (though "cloud" and "platform" are pretty vague too).
Like so many things in computing, we like to make up words and also reuse old ones for completely different things.
I'd say "hosting" means making something available to you in the cloud. Model makers don't offer that.
They sell "inference," which is a compute task with real hardware costs for them. The task runs against their model, which is not hosted for you.
Are you really hosting the model? Are you making its bed, cooking its food and cleaning up for it?
Inference sounds cooler and more mystical. Any old company can host something, "running inference" requires 10x rock star engineers and a CEO that acts like a badass who wears a leather jacket and has Thoughts about demographics.
This literally has to be it. The people that say it tend to say it a lot.
Same difference between S3 bucket and Google Photos (web). Both "host" images but one provides an experience on top of the underlying assets.
That's interesting, the idea that hosting implies a front-end (whether UI or API).
So it's similar to why we don't call Heroku or Vercel "hosting" or "compute" because it's more of a service (though "cloud" and "platform" are pretty vague too).