最新模型
Axion Releases a Three-Stage Agent Training Recipe; Downloaded Weights Require Separate Evaluation
Two rounds of supervised fine-tuning followed by reinforcement learning, with inspectable data and training workflows. The linked 35B weights are a community reproduction, and paper scores should not be treated as the downloaded model’s performance.
If you want to train an AI agent that can search for information and run calculations, Axion 1.0 offers a post-training recipe that can be examined stage by stage. Researchers from ByteDance, Duke, the University of Maryland, and the University at Buffalo have released code for data generation, filtering, training, and evaluation. The repository is marked as licensed under Apache 2.0. For teams preparing to retrain a model, the value lies in being able to inspect how its capabilities develop, rather than just seeing a scorecard. Original repository
Before downloading a model, check which version it is: the 35B weights linked from the official repository are explicitly labeled as a community reproduction, not an official Axion checkpoint, and they have a separate set of evaluation results. Version notes
What each of the three stages does
Axion first runs two rounds of supervised fine-tuning (SFT), teaching the model from worked solution demonstrations, then uses reinforcement learning (RL) to adjust its behavior based on interaction outcomes. This post-training process does not add a dedicated mid-training stage, preference-pair training with DPO, or on-policy distillation. That does not mean the base model had no other prior training. Process overview
The first stage uses 10,819 science and math solution trajectories, making problems the base model can already solve progressively harder. The model can use Python to solve them, but cannot search the web. The second stage uses 7,544 tool-task trajectories: programs first filter the data and calculate reference answers, then a language model turns them into questions. Questions that can be answered correctly without retrieval are excluded. Data construction notes
The two fine-tuning rounds therefore have distinct goals: first practice independent reasoning, then practice gathering external evidence. The subsequent RL stage lets the model interact with search, web browsing, academic retrieval, and Python tools. A correct answer earns the full reward, while verified intermediate progress can earn partial reward. Training recipe
Scores improve in some areas and decline in others
The authors’ stage-by-stage results show the 35B model’s score on the information-search benchmark SEAL-0 rising from 44.1% for the base model to 53.2% after the two SFT rounds, then to 62.2% after RL. But on the FrontierScience Research scientific reasoning benchmark, the score fell from 58.3% before RL to 56.7% afterward. The gains did not appear equally across tasks. Technical report, Table 2
These results come from evaluations in an agent environment where the model could search, browse, and execute Python, and remain subject to each benchmark’s tool constraints. Most benchmarks were run three times, with the best single-run score reported rather than the average; BioMysteryBench was run five times. Stage-by-stage scores can therefore show how this particular setup changed, but do not prove that other training stages are generally useless. Evaluation methodology
Reproducing the weights and retraining are different decisions
The public reproduction uses a mixture-of-experts architecture, in which only some parameters are active in each computation. Its model card lists about 35 billion total parameters, 3 billion active parameters, and BF16 weight precision. The release page reports 60.4% on SEAL-0 and 55.0% on FrontierScience Research, both below the paper’s listed scores of 62.2% and 56.7%. These figures are reported by the community publisher; this article did not run tests, so the gap cannot be attributed directly to any particular training stage. Reproduction model card
If your goal is retraining, the small number of stages alone does not imply low cost: the published 122B SFT recipe uses 32 GPUs and a 128K context. Axion provides a workflow that can be inspected, but downloaders should assess suitability using the reproduction’s results, while teams retraining should estimate the investment based on their specific data and hardware setup. Configuration notes
Sources
- GitHub - bytedance/axion1_0: Frontier-level agent capabilities with nothing more than high-quality data and a simple training recipe: 2 SFT + RL, no mid-training/DPO/OPD. Full rec…
- axion1_0: Frontier-level agent capabilities with nothing more than high-quality data and a simple training recipe: 2 SFT + RL, no mid-training/DPO/OPD. Full recipe released.
- Axion 1.0: Open Agent Foundation Models, Simplified
- Axion1-0/Axion-1.0-35B · Hugging Face