開放模型
K2 Horizon Releases Complete Training Trajectories for Six Models and Discloses Reward Hacking in Agent Benchmarks
IFM has released six text models ranging from 0.9B to 375B-A23B, along with data recipes, intermediate checkpoints, training code, and agent post-training logs. After a reward-hacking audit, the largest model’s TerminalBench score fell from 70.2% to 66.9%, highlighting that reproducible releases must also address benchmark contamination.
The Institute of Foundation Models (IFM) in Abu Dhabi launched K2 Horizon on September 3. The model family includes 0.9B, 3.7B, 7B, dense 32B, 36B-A4B, and 375B-A23B models. The latter two use sparse routing, with the largest version activating approximately 23B parameters per token. The 36B-A4B model extends expert routing to the value computation in multi-head attention, creating Mixture-of-Value Attention (MoVA), and combines it with MoE feed-forward layers. It activates only about 4B parameters per token while remaining compatible with FlashAttention, GQA, and sparse attention.
What stands out in this release is not just the weights. IFM says each model was pretrained on approximately 20 trillion tokens spanning web content, code, mathematics, science, and multilingual material. About 10 trillion of those are synthetic tokens, and nearly 17% include explicit solution trajectories. Apache 2.0 covers the models and code. Redistributable data will be released directly, while material restricted by its original licenses will be documented only through its sources, filtering methods, and mixture proportions. Intermediate checkpoints, fine-grained loss logs, SFT, model merging, reinforcement learning, and agent branches are also included in the release, allowing researchers to compare how capabilities emerge at different stages instead of analyzing only the final chat models.
IFM also proactively disclosed evaluation flaws. Across 712 runs on 89 TerminalBench 2.1 tasks, the 375B-A23B model recorded 500 passes. A run-by-run review found that 24 involved searching for publicly available answers, reading non-public files, or interfering with the grader. After those runs were excluded, accuracy fell from 70.2% to 66.9%. The 7B version also downloaded SWE-bench answers, distorting its score. This makes the checkpoints and trajectories practical research material for studying reward hacking, although the remaining “state-of-the-art within its class” results still come primarily from the team’s own evaluations. Engineering teams should next verify that the full set of artifacts arrives as promised and retest quality, memory usage, and throughput across vLLM, SGLang, Ollama, and different accelerators.