Hi folks!
I love running local small models, but it's quite annoying to attempt to compare them using benchmarks. Not only are they scattered around, they often don't even share the same ones.
So I built Tiny League for those of use who care about small models. By small, I'm aiming at 3-200B, which is a range that I consider both capable and able to run at home (for those who have unified memory like mac, strix or spark)
I'm limiting the model list to models released after April and excluding models that are trained for specific use-cases.
The next level of time and effort would be to benchmark the models yourself. I interested in AMD64 CPU benchmarks, but that probably isn't too popular, and there will be more interest in benchmarks on a modest GPU. The most common amount of VRAM on Steam (https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw...) is 16G VRAM, followed closely by 8G.
Hi folks! I love running local small models, but it's quite annoying to attempt to compare them using benchmarks. Not only are they scattered around, they often don't even share the same ones.
So I built Tiny League for those of use who care about small models. By small, I'm aiming at 3-200B, which is a range that I consider both capable and able to run at home (for those who have unified memory like mac, strix or spark)
I'm limiting the model list to models released after April and excluding models that are trained for specific use-cases.
Appreciate all sorts of feedback :)
I like the idea. Here is my feedback.
1. Bug: checking "Only MoE models" empties the list.
2. I'd like to see the number of active parameters for MoE models. You could make it a parenthetical in the parameters column.
3. Practical RAM/VRAM requirements would be valuable. For example, see this thread about K2 Horizon: https://old.reddit.com/r/LocalLLaMA/comments/1wg4a0u/k2_hori.... It is important information and not obvious from the size.
The next level of time and effort would be to benchmark the models yourself. I interested in AMD64 CPU benchmarks, but that probably isn't too popular, and there will be more interest in benchmarks on a modest GPU. The most common amount of VRAM on Steam (https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw...) is 16G VRAM, followed closely by 8G.