開放模型
Kimi K3 Releases 2.8T MoE Weights, With 104B Active Parameters Raising the Self-Hosting Bar to Cluster Scale
Moonshot AI has officially released the Kimi K3 weights and technical report. The model features native vision, a one-million-token context window, and activates 16 of 896 experts for each token. The open weights finally make the company’s capability claims reproducible, but more than 1 TB of low-precision weights still put full deployment beyond ordinary workstations.
Moonshot AI released the Kimi K3 model weights and technical report on July 27, delivering the core assets that were still missing when the product launched in mid-July. K3 is a sparse mixture-of-experts model with 2.8 trillion total parameters and 104 billion parameters activated per token. Its router selects 16 of 896 experts, with additional shared experts. The model integrates the MoonViT-V2 vision encoder, can process text, images, and video, and has a stated context limit of one million tokens.
The architecture is notable for more than its scale. Kimi Delta Attention compresses some historical information into a finite state, reducing attention costs for long sequences. Attention Residuals allow different layers to directly mix earlier attention outputs instead of relying solely on conventional layer-by-layer residual propagation. Moonshot also stores most of the MoE weights in MXFP4 to reduce memory traffic and deployment costs. These designs make the compute required per step far lower than for a dense 2.8T model, but they do not eliminate the storage capacity and inter-GPU communication needed to load the full set of experts.
The technical report claims that K3 outperforms several open and existing commercial models on long-horizon software development, terminal tool use, visual-agent tasks, and general-knowledge benchmarks, while still trailing Claude Fable 5 and GPT-5.6 Sol overall. These comparisons were conducted primarily by Moonshot, and the choice of agent harness, inference budget, and tool environment can significantly affect the results, so no single leaderboard should be treated as definitive.
For engineering teams, the real news is that inspectable, quantifiable, and modifiable weights are now available—not merely another model name in an API. The next things to watch are formal support in runtimes such as vLLM and SGLang for KDA, visual inputs, and expert parallelism, as well as whether the community can reproduce the long-horizon agent results using the same harness. Although the license permits broad commercial use, self-hosting teams must still estimate the real costs of more than a terabyte of weights, inter-node bandwidth, and the KV state required for a one-million-token context window.