開源模型與推論
K2 Horizon Launches in Six Sizes, With Its MoVA Model Activating Only a Subset of Value Projections per Token
IFM has released the K2 Horizon model family, ranging from 0.9B to 375B parameters. The 36B version extends MoE-style routing to the attention mechanism’s value projections. Weights and deployment recipes are now available, but the organization’s claim of being “fully open source” still runs ahead of what it has actually delivered.
The Institute of Foundation Models (IFM) has released K2 Horizon in six sizes: 0.9B, 3.7B, 7B, 32B, 36B-A4B, and 375B-A23B, under the Apache 2.0 license. Beyond spanning use cases from edge devices to multi-GPU servers within a single family, the 36B-A4B model is particularly noteworthy. It combines Mixture-of-Experts with Mixture-of-Values Attention (MoVA), using a router to select a subset of value projection matrices for each token instead of having all tokens share a single value projection. According to the model card, it stores roughly 36B parameters, activates about 4B per token, and natively supports a 524,288-token context window.
Deployment paths are available for Transformers, vLLM, and SGLang. IFM’s validated SGLang recipe uses two H200 GPUs, BF16, TP=2, EP=2, and FlashAttention-3, and also requires a specific router GEMM split configuration to preserve numerical behavior. For engineering teams, this is a reminder that “activating only 4B” describes per-token compute, not a requirement to load only 4B parameters. Nor does it mean that a typical consumer GPU can easily handle a 512K KV cache. Results reported in the model card for Terminal-Bench 2.1, tau3-Banking, and other benchmarks currently come primarily from the publisher and still need to be reproduced by third parties using the same sampling and inference settings.
There is also a timing gap in the release’s transparency. IFM’s press release claims that the weights, code, training data, and methods have all been fully released. However, the 36B model card explicitly says that intermediate checkpoints, data, and training code “will be released.” At present, the final weights, model implementation, and some post-training resources are directly downloadable. Researchers should next verify whether the data recipe, complete pretraining code, and intermediate checkpoints actually become available, and test whether MoVA maintains quality and throughput across different inference frameworks, quantization formats, and long-context workloads.