The same LLM is 8x slower to first token depending on who serves it

3 points | by jeremyrajan an hour ago

1 comments