Yeah! So I did teams from a fixed, random seeded pool and then would rerun it with the switched teams. It also doesn't play against ladder yet but I played against it and it's not horrible. It plays better than an amateur but I was able to beat it a good amount
what was the selection for the teams? random per round? how does it play against people in ladder?
Yeah! So I did teams from a fixed, random seeded pool and then would rerun it with the switched teams. It also doesn't play against ladder yet but I played against it and it's not horrible. It plays better than an amateur but I was able to beat it a good amount
Pokémon Showdown isn't a game so much as it is competitive Wiki reading.
Not going to lie I agree lol but still interesting to see
Jev will give better results on this kind of thing.
For sure are many better models for this use case[0]
At the same time, it’s interesting seeing how generally capable language models/LLMs/agents/harnesses are getting
We’ll probably see a lot more experiments and benchmarks like these in the near future
0: simple classifiers can play quite a few game, including Doom https://playground.jeffyclassify.com/#doom
Yeah Jev can play it but Jev cant communicate with its teammate in the 2v2 + it cant reason and is just a decision model