代理模型與後訓練
NeoHorse-1 Feeds Agent Execution Traces Back into Model Training, but “Recursive Self-Improvement” Remains a Single-Round Prototype
NeoHorse-1 uses a router to collect traces from different models completing agentic tasks, then trains 4B and 9B models through curriculum-based SFT and on-policy distillation. Its official aggregate scores exceed those of same-size Qwen3.5 baselines, but it has yet to demonstrate that repeated rounds of self-updating can deliver sustained gains or prevent data drift.
TokenRhythm released the 4B and 9B NeoHorse-1 weights on September 8, along with a [technical report](https://arxiv.org/abs/2609.08183) and [project repository](https://github.com/TokenRhythm/NeoHorse). Both models were created by post-training same-size Qwen3.5 variants. The aim is not merely to improve conversational scores, but to preserve the tool calls, reasoning, and harness context required for agent execution. The weights are licensed under Apache 2.0, with BF16 and 8-bit, 5-bit, and 4-bit GGUF variants also available for self-hosting through vLLM, SGLang, llama.cpp, or compatible tools.
At the core is a routing harness. The system first estimates the capability requirements of each request and selects a service tier from a heterogeneous model pool, while recording the selection, tool interactions, and outcomes. Before entering a three-stage curriculum-based supervised fine-tuning pipeline, traces undergo exact and approximate deduplication, evaluation-contamination filtering, structural validation, six-dimensional semantic scoring, and Scene/Goal/Outcome sub-scenario labeling. The same routing signals are also used for on-policy distillation, in which a teacher supervises responses generated by the student itself. Subsequent evaluation results then determine the capability mix of the next training batch, forming an “evaluate—select—update” feedback loop.
The team reports that the 4B model’s macro-average across agentic, tool-use, coding, and instruction-following evaluations rose from 58.94 to 64.87, while the 9B model improved from 65.60 to 69.04. The 4B model scored 88.46 on tau2-Bench and 59.43 on LiveCodeBench v6. These figures indicate that execution traces can serve as effective post-training data, but they are insufficient to support the stronger claim of “recursive self-improvement.” What has been released so far is a checkpoint produced by a single post-training round; there are no experiments covering continuous updates across multiple generations, cost curves, capability degradation, or drift in safety behavior. Engineering teams should next examine whether the router favors expensive teacher models, how failed traces enter the dataset, and whether the closed loop can preserve its gains on unseen tasks after continued operation.