DeepInfer, our in-house inference framework, now open source

3 comments

• an hour ago
  [deleted]
• pluc an hour ago

  https://news.ycombinator.com/item?id=50163277

• awasum_yannick an hour ago

  This probably makes enterprise private inference cheaper. Wonder if this can be generalized to server other LLMs not just DeepSeek...should not be too hard