開放權重模型
Kimi K3 Pushes Open-Weight Models Toward the Three-Trillion-Parameter Scale with a 2.8T-Parameter MoE
Moonshot AI has released Kimi K3, featuring a Mixture-of-Experts (MoE) architecture with 2.8 trillion total parameters and 16 experts activated per token, along with support for a one-million-token context window and visual inputs. The full weights are scheduled for release later, so API performance should still be distinguished from evidence based on self-hosted deployments.
Moonshot AI launched Kimi K3 on July 16, positioning it as an open-weight model approaching the three-trillion-parameter scale. Publicly available information indicates that the model has 2.8 trillion total parameters and 896 experts, with 16 selected for each token, and offers an approximately one-million-token context window and native vision capabilities. Its architecture incorporates Kimi Delta Attention, a hybrid linear-attention mechanism, as well as Attention Residuals, which modify how information is passed between layers. During post-training, Moonshot also introduced quantization-aware training, targeting formats that include MXFP4 weights and MXFP8 activations.
MoE's low activation ratio can reduce per-token computation to far below what the total parameter count might suggest, but it does not eliminate memory, communication, or deployment challenges. A system with 896 experts requires efficient routing, load balancing, and all-to-all communication. If inter-node bandwidth is insufficient, communication stalls may negate the theoretical advantages of sparse computation. A one-million-token context window also cannot be evaluated solely by its maximum length; long-sequence retrieval, positional bias, KV cache usage, and the consistency of multi-turn agent state must also be measured.
Programming and agent benchmarks published by Moonshot show K3 approaching some proprietary frontier models and performing particularly well in blind frontend coding tests, but most of the figures still come from the developer or a single leaderboard. The actual availability date of the full weights, licensing terms, quantized checkpoint quality, and minimum cluster requirements will determine whether the model can truly be self-hosted at scale. Researchers should also examine expert utilization, capability gaps across languages, and the compatibility of low-precision formats with hardware other than the latest generation.