Seriously, what's going on there ? Why is it so different from others? Is it just behind technologically/training wise or it's using something fundamentally different?
It performs much better than composer2.5 (while being as fast). It's not Opus, but I think it's not that far off. Definitely better than sonnet for what I've been doing.
On the other hand, I think they probably heavily adapted the training data so that it really is extremely focused on code. I just recently ran my personal "poetry benchmark" on it (where I give it ~850 poems I've written over my life and ask it to comment the corpus as a whole), and it's whack. It tries to write in portuguese (most of the poems are portuguese) and code-switches constantly and mixes up words to the point of making what it writes almost unreadable (e.g. it writes stuff like "You can't QoS that that look for beast poems", in portuguese, all messed up). The quality of the analysis is also quite bad (I'd say it's definitely behind Sonnet).
So I really think they either threw away data that wasn't tied to coding so that they could fine-tune it to that, or somehow they've got such an unbalanced dataset that coding ends up dominating either way. To me, its disastrous performance in this drawing "competition" fits this narrative.
Did Elon tell some poor engineer to give grok a prompt injection for drawing,"make it look like one of my childhood drawings!" Just like the Tesla truck?
Claude was clearly 'pushing back' on the coziness of the cabin. But I think it did best with the cat. Grok, I fear, is making a case for euthanasia. It's suffering and I think it would be cruel to let it continue. Someone pull the plug. .
If it is, it's a horrible ad. My main takeaway is that all of them were god awful at image generation and Fable was 20x more expensive and still awful.
Not useless. LLMs are the most general purpose computer algorithms ever created. They are getting smarter and cheaper at a geometric rate. What is a bad idea today could have useful applications tomorrow.
Citation needed - my company is paying more than ever for code generation. I have no reason to believe (given anecdotes) that anyone finds themselves in the opposite situation.
Grok! LOL!
Seriously, what's going on there ? Why is it so different from others? Is it just behind technologically/training wise or it's using something fundamentally different?
Grok 4.5 is...something else.
It performs much better than composer2.5 (while being as fast). It's not Opus, but I think it's not that far off. Definitely better than sonnet for what I've been doing.
On the other hand, I think they probably heavily adapted the training data so that it really is extremely focused on code. I just recently ran my personal "poetry benchmark" on it (where I give it ~850 poems I've written over my life and ask it to comment the corpus as a whole), and it's whack. It tries to write in portuguese (most of the poems are portuguese) and code-switches constantly and mixes up words to the point of making what it writes almost unreadable (e.g. it writes stuff like "You can't QoS that that look for beast poems", in portuguese, all messed up). The quality of the analysis is also quite bad (I'd say it's definitely behind Sonnet).
So I really think they either threw away data that wasn't tied to coding so that they could fine-tune it to that, or somehow they've got such an unbalanced dataset that coding ends up dominating either way. To me, its disastrous performance in this drawing "competition" fits this narrative.
Did Elon tell some poor engineer to give grok a prompt injection for drawing,"make it look like one of my childhood drawings!" Just like the Tesla truck?
Claude was clearly 'pushing back' on the coziness of the cabin. But I think it did best with the cat. Grok, I fear, is making a case for euthanasia. It's suffering and I think it would be cruel to let it continue. Someone pull the plug. .
Fix this signup error please
https://www.tryai.dev/?error=server_error&error_code=unexpec...
I think this is just an ad.
If it is, it's a horrible ad. My main takeaway is that all of them were god awful at image generation and Fable was 20x more expensive and still awful.
Well, your takeaway is bad. This isn't image generation, it's tool use with a digital paintbrush.
Useless They are not image generation models.
Not useless. LLMs are the most general purpose computer algorithms ever created. They are getting smarter and cheaper at a geometric rate. What is a bad idea today could have useful applications tomorrow.
> cheaper at a geometric rate
Citation needed - my company is paying more than ever for code generation. I have no reason to believe (given anecdotes) that anyone finds themselves in the opposite situation.
That's what makes it an interesting challenge.
Tell them to draw LeBron James or Heisenberg. Everyone will refuse.