My gut reaction was that people are continuing to do random stuff with Jev just because it's the hot new thing, but you preempted pretty much all my objections in the post. Nice experiment.
> So for now, at least on this task, the hosted model is both far more accurate and, given the cost above, hard to beat on price.
It's probably still worthwhile comparing to a frontier model, although you presumably wouldn't want to do that with sensitive data.
My gut reaction was that people are continuing to do random stuff with Jev just because it's the hot new thing, but you preempted pretty much all my objections in the post. Nice experiment.
> So for now, at least on this task, the hosted model is both far more accurate and, given the cost above, hard to beat on price.
It's probably still worthwhile comparing to a frontier model, although you presumably wouldn't want to do that with sensitive data.