The Htop for LLM Inference

3 points | by helasaoudi 2 hours ago

1 comments

  • rbanffy 33 minutes ago

    This is super cool. Would be nice if it could be adapted to other architectures besides Nvidia. How is it getting its data?

    BTW, exporting the data to Prometheus would be a killer feature.