Kimi-K3 Technical Report [pdf]

187 points | by vinhnx an hour ago

37 comments

  • m_ke 42 minutes ago

    Also open sourced a bunch of infra to go with it.

    Anyone who claims open source and open weights models are "decel" needs to get their head checked

    https://github.com/MoonshotAI/MoonEP

    https://github.com/kvcache-ai/AgentEnv

    https://github.com/MoonshotAI/FlashKDA

      jvanderbot 41 minutes ago

      This comment would be much better without the second line

        ike_a 35 minutes ago

        I'm not sure I understand the case for open-source models being decelerationist, is this it?

        Decel:

        - Potentially reduces investor appetite for funding big labs.

        - More risk of powerful AI getting in bad hands -> more regulation.

        Accel:

        - More competition so big labs can't rest on laurels.

        - More research in open, so all labs can accrete advancements faster.

        I feel like open-source = acceleration has a much more clear argument. (and how bad would deceleration be in any case?)

          StevenWaterman 31 minutes ago

          I think it's basically open weights => more inference competition => less profit from inference => less training competition

            f311a 9 minutes ago

            > less training competition

            I think you meant less research and experiments in big labs because they don't get all the AI money.

            Training is expensive, but they also have more than 10 000 of employees combined and they cost a lot of money.

          Iolaum 31 minutes ago

          Open Source models decelerate growth of closed AI. For people who think (or want) AI = closed_AI then that argument has weight. Good luck getting them to update their priors.

          zozbot234 22 minutes ago

          Open source AI is actually a lot less "powerful" than genuine frontier models, i.e. it has a much tighter inherent capability ceiling. This is "decelerationist" from a purely AGI-pilled point of view but is actually great if you're worried about a capabilities arms race putting AI Safety at severe risk.

          Kimi K3 is plausibly a lot less dangerous than a totally jailbroken ChatGPT/Gemini/Claude Sonnet (let alone Opus or Fable!) and it's quite deeply weird how no one seems to be calling for those models to be banned or restrained by further regulation. Why the double standard against the less concerning (but more efficient!) open weight models?

        acedTrex 34 minutes ago

        Why?

  • fahrradflucht 39 minutes ago

    License: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE

    > If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.

    + the existing 100 million monthly active users, or more than 20 million US dollars for commercial products have to name Kimi clause

      simonw 12 minutes ago

      Kimi 2.6 had the same janky license: https://huggingface.co/moonshotai/Kimi-K2.6/blob/main/LICENS... - looks like they've been doing that at least as far back as K2.

      The models they released in 2025 - https://huggingface.co/moonshotai/models - were clean MIT. They started doing the "modified MIT" thing in January 2026 with moonshotai/Kimi-K2-Thinking

        zozbot234 6 minutes ago

        To be clear, the restriction on "Model as a Service" past $20M yearly revenue is new to K3. K2.x had the attribution requirement for any commercial use with more than $20M monthly revenue.

          simonw 5 minutes ago

          Good catch, thanks.

      embedding-shape 8 minutes ago

      Hah, so much for "open weights" :D Fair enough, they're at least downloadable, shame they didn't end up being actually open, nor open source, the community had really high hopes for this. But again, still available for download, so better than nothing else I suppose.

        jhonof 4 minutes ago

        Open source projects frequently have separate commercial licenses no? This isn't abnormal.

      throwaway27448 29 minutes ago

      I wonder how they'll figure out who to target for litigation when this license is violated.

      NitpickLawyer 28 minutes ago

      Interesting license. They basically added a "no composer / better models" if revenues (including downstream) above 20m clause, where you'd have to get a separate agreement with them.

      Still has the attribution clause if serving this to your users.

      Other than that, free to do whatever you want.

  • eamag 24 minutes ago

    > we build a self-evolving, hierarchically organized knowledge graph that agents continuously expand through web-scale exploration across knowledge-intensive and coding domains

    That's interesting!

  • a-dub 28 minutes ago

    knowledge graph guided task synthesis. very cool! i have long wondered about the "how do you get good coverage of all the tasks" problem.

    maybe some interesting theoretical work there around the rate of production of new knowledge itself and various mechanisms (human approaches, mechanistic approaches, etc).

  • storus an hour ago

    What would be the current best method to fine-tune it for my own specific agentic tasks? LoRA + DPO? GRPO? Something else?

      whimsicalism an hour ago

      LoRA + SFT, but it'll be big - better to wait for a finetuning API from one of the providers, I wouldn't jump straight to RL or off-policy pseudo-RL like DPO.

  • brcmthrowaway 2 minutes ago

    Is this p-hacking?

  • eamag 26 minutes ago

    Can someone explain what are teachers in Multi-Teacher On-Policy Distillation? I can imagine math, coding and other verifiable domains, but they also have biology? Is it where distillation from bigger models come in?

  • whimsicalism 31 minutes ago

    It's funny that we've finally returned to tanh activation functions, time is a circle.

  • lenerdenator 17 minutes ago

    What would it take to get an American open model to compete with this?

      embedding-shape 5 minutes ago

      Latest "big" release from any of the bigger American lab must have been GPT-OSS-120b I think? Released ~summer 2026, so pretty much a year ago. Doesn't seem like it'll happen by itself, so something either forcing their hand figuratively, or something forcing their hand literally.

      chrsw 8 minutes ago

      Something beyond my imagination

      boomskats 10 minutes ago

      An act of G̶o̶d̶ Congress?

  • colesantiago 25 minutes ago

    This is amazing to witness. Moonshot open sourcing Kimi K3, a frontier AI and other components really means we are getting abundant AI for all of humanity.

    Kudos to Moonshot for truly being what OpenAI should have been.

    Fable-level and frontier AI should be open source and available to everyone for free.

      embedding-shape 4 minutes ago

      > Kudos to Moonshot for truly being what OpenAI should have been.

      Kudos to Moonshot for making these weights available for download. Lets not fool ourselves and claim these are "open source" by any understanding of the concept though, there are usage restrictions (even if you download them) and also training data isn't clearly broken down either, nor it it actually using a FOSS license.

  • m00dy an hour ago

    I would want to see three things before drawing strong conclusions:

    End-to-end tokens/sec and cost on realistic coding agent trajectories, including tool outputs and retries, not isolated decode benchmarks.

    Cache hit rates and prefill cost for branching, multi-turn sessions.

    Router-load distributions after post-training, where expert collapse or specialization problems often show up.