LLM continuous batching host scheduler in Rust (191k tok/s on 1 vCPU)

1 points | by bmartin-systems 43 minutes ago

No comments yet