31 points | by nojs an hour ago
4 comments
They realized Astra having the same score as Sol was silly so they rushed to update the index so it fits what people expect.
The old index was clearly bad (Astra is way better than Sol) but it's also unscientific to tweak it like this.
What version the intelligence vs cost graph is using? they didn't ran v4.2 to all models.
Why have they not included ARC-Agi-3 on their index?
Clearly that would move things around.
How to view the previous version to compare?
They realized Astra having the same score as Sol was silly so they rushed to update the index so it fits what people expect.
The old index was clearly bad (Astra is way better than Sol) but it's also unscientific to tweak it like this.
What version the intelligence vs cost graph is using? they didn't ran v4.2 to all models.
Why have they not included ARC-Agi-3 on their index?
Clearly that would move things around.
How to view the previous version to compare?