模型訓練／程式代理
T1 Replays MoE Routing to Stabilize Long-Horizon Reinforcement Learning, Raising Terminal-Bench 2.1 Success Rate to 64%
T1 records and replays MoE expert selections for every token, preventing numerical differences between rollout and training from sending them through different subnetworks. Its open weights support self-hosting, but benchmark results and competitor comparisons still come mainly from the team’s own evaluations.
Tencent’s Hunyuan team and researchers from several universities have released T1, a terminal-agent model post-trained from Qwen3.5-122B-A10B. It has roughly 122B total parameters, activates about 10B per token, and can execute more than 300 consecutive tool calls in a cloud sandbox. The team also released the model weights, technical report, and training-data artifacts, allowing outside researchers to examine this approach to long-horizon agent reinforcement learning rather than seeing only API results.
The central problem is the training–inference mismatch in Mixture-of-Experts (MoE) models. As an agent performs a task, the conversation framework repeatedly retokenizes the text history. Even small numerical differences can change the router’s top-k expert selections, causing PPO to update a policy different from the one that generated the trajectory. T1’s TITO mechanism directly preserves the token IDs and log probabilities from sampling, then corrects offsets caused by retokenization at turn boundaries. R³ records the experts actually selected at every layer for every token and replays the same selections during the training forward pass, while still computing gate scores with the current parameters so the router remains trainable. Together, the two mechanisms reduced the team’s measured mean log-probability gap from 0.021 to 0.013.
The reward design also moves beyond pass-or-fail scoring for the entire task, instead counting the number of assertions passed in the verifier. The researchers first warm-started a full-size critic, then used PPO to train on the audited T1-15k task set. Across 89 held-out Terminal-Bench 2.1 tasks, the base model, SFT model, and T1 achieved success rates of 43.8%, 49.4%, and 64.0%, respectively. T1 scored 27.9 on Long-Horizon Terminal Bench.
These figures should not be treated as a general-purpose agent ranking. The authors ran the comparisons using a unified harness, and Claude Sonnet 4.6 still scored 37.3 on the long-horizon benchmark, showing that rankings on the shorter Terminal-Bench cannot be extrapolated to longer tasks. The public checkpoint also contains roughly 125B parameters, and no hosted inference provider is available, so practical reproduction requires substantial GPU memory, sandbox capacity, and security isolation. The next questions are whether third parties can reproduce the PPO gains and whether TITO and R³ can be transferred to other MoE models and agent frameworks.