21 points | by mike-the-brain an hour ago
4 comments
Great news, has made low memory bandwidth model usage so much nicer.
Amazing tech
> An agent writes in an afternoon what a chatbot writes in a month
But can you just.. not.
Your tech is so good, it speaks for itself. Don't ruin that.
I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.
vllm PR for DFlash2: https://github.com/vllm-project/vllm/pull/52816
Great news, has made low memory bandwidth model usage so much nicer.
Amazing tech
> An agent writes in an afternoon what a chatbot writes in a month
But can you just.. not.
Your tech is so good, it speaks for itself. Don't ruin that.
I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.
vllm PR for DFlash2: https://github.com/vllm-project/vllm/pull/52816