代理訓練
MidTool Releases 20.3B-Token Tool Mid-Training Dataset; 4B and 8B Models Improve Across Three Agent Benchmarks
MidTool places APIs, MCP, documentation, code, and synthetic workflows in a dedicated stage after pretraining and before SFT. The dataset and model checkpoints are now public, but the models still score zero on deep-search evaluations, showing that learning schemas is not the same as acquiring long-horizon research capabilities.
MidTool treats general tool use as a foundational capability that should be developed during mid-training, rather than added later through a small amount of SFT or reinforcement learning trajectories. The team’s publicly released MidTool-Mix contains approximately 20.3 billion tokens across 11.22 million samples: 42% comes from technical web pages, 26% is code, 23% is sourced from PDFs, and the remaining 9% consists of native agent trajectories. Hugging Face also hosts the dataset, 4B and 8B mid-training and RL checkpoints, and the FastText classifiers used to filter web pages and PDFs.
The data pipeline first collects material from FineWeb, FinePDFs, GitHub, REST API definitions, MCP skills, and documentation-based workflows, then cleans each source separately. Code from benchmark repositories is excluded to reduce contamination, while a classifier trained on teacher-generated labels filters text for documentation-dense content. MinHash LSH is then used for deduplication. The synthetic stage follows two paths. One extracts tool affordances from documentation to generate context-grounded question-answer pairs and multi-turn interactions. The other builds single-call tasks, multi-tool plans, and insufficient-information scenarios directly from tool schemas, forcing models to learn how to select tools, populate parameters, order steps, and request clarification.
The researchers continued training Qwen3-4B-Base and Qwen3-8B-Base on MidTool-Mix, then applied the same SFT or RL procedures. Compared with baselines that used post-training alone or generic mid-training corpora, both model sizes improved on BFCLv3, τ²-bench, and MCP-Universe. The 4B model’s Pass@1 score on τ²-bench was nearly double that of the SFT-only baseline, and it also began subsequent SFT with a lower loss. However, it still scored 0.00 on MCP-Universe’s deep-search subset, indicating that parameter grounding and transfer to short workflows have not yet translated into iterative evidence gathering and long-horizon control. Engineering teams should next examine dependence on teacher models, synthetic-data bias, benchmark-contamination audits, and whether the cost of training on 20.3 billion tokens delivers better value than a smaller, targeted SFT dataset.