代理訓練與評測
FACET Builds an Executable Environment Before Generating Terminal Tasks, Boosting the 9B Agent’s Score by 8.24 Points
FACET binds instructions, containers, reference solutions, and verifiers to the same execution state, synthesizing 6,078 verified tasks from more than 71,000 agent skills. Fine-tuned on just 1,200 successful trajectories, Qwen3.5-9B improved its Terminal-Bench 2.1 score from 27.34 to 35.58.
Synthetic data for terminal agents typically contains four interdependent components: user instructions, a preconfigured environment, a reference solution, and an executable verifier. If separate generation stages independently infer files, dependencies, or data formats, a task may be impossible to complete or graded incorrectly even when its text appears reasonable. FACET, released on August 19, instead begins by reconstructing the objective, tool constraints, intermediate states, and input/output contracts from relevant Agent Skills, then builds and repairs the Docker environment. All subsequent components read from the same realized state, and when a failure occurs, only the problematic component is regenerated.
The study created 7,852 scenario seeds from 71,341 valid skills, ultimately producing 6,078 execution-verified tasks. Each task has an average of 22.77 tests, providing denser test coverage than the other terminal datasets listed in the study. The authors used DeepSeek-V4-Pro to generate execution trajectories for roughly 6,000 tasks, selected 1,200 fully successful trajectories, and conducted three rounds of full-parameter supervised fine-tuning on Qwen3.5 4B, 9B, and 27B. Under the same Terminus-2 framework and three-attempt setting, their Terminal-Bench 2.1 scores increased by 7.12, 8.24, and 6.75 points, respectively. The 27B version reached 47.57, just 1.49 points behind the 397B model’s score of 49.06 under the same setting.
Ablation experiments also showed that sequentially building the environment, instructions, solution, and solution-aware verifier achieved a final yield of 83%, compared with 65% for joint generation and 63% for reverse generation. However, this was not a strictly univariate comparison, as the workflows also differed in their information flow, validation, and repair strategies. Not all leaderboard reference models were rerun under exactly the same framework, either. The code, 6,020 public tasks, and three checkpoints are now available, but the complete trajectories used in the paper and the generation costs still require further verification.