AI 代理與研究自動化
DisCo Distills 1,000 Machine Learning Repositories Into 5,353 Verifiable Agent Skills
New research turns source code, documentation, and tests into operational knowledge that can be loaded on demand, delivering gains across four research benchmarks with the same underlying model and agent framework. Skill-building costs are accounted for separately, however, and the results remain the authors’ own evaluation, so they should not be treated as evidence that skills can replace model training at low cost.
Research agents often already have a harness responsible for planning, memory, and tool calling, yet they must still rediscover framework APIs, data formats, and debugging techniques during every task. The [Repo-To-Skill paper](https://arxiv.org/abs/2609.02749) defines this kind of implementation know-how as “operational knowledge” and introduces DisCo, which first distills skills from repositories or papers and then retrieves them on demand during research tasks, rather than placing an entire project in the context window.
Each repository is converted into a skill graph. `SKILL.md` describes when a skill should be used, `references/` preserves supporting evidence, and `scripts/` provides executable workflows. Candidate skills must be validated using native examples, tests, CLI checks, or small smoke tests; failed skills can be repaired locally and rerun. The public [AREX-Skill](https://github.com/VectorSpaceLab/AREX-Skill) snapshot contains graphs for 1,000 repositories and 5,353 skills spanning 20 domains and 178 capability families. A two-stage router first narrows the candidate set before the agent loads the relevant skills, helping control context costs. The format can be integrated with coding agents including Codex, Claude Code, and Pi.
The study compared runs with and without skills while holding GPT-5.5, the Codex harness, reasoning effort, and downstream execution budget constant. Relative performance improved by 134.3% on MLE-bench and 34.4% on PaperBench, while FrontierCS and PassNet improved by 9.2% and 14.0%, respectively. For PaperBench, the target papers and their implementations were excluded; MLE-bench likewise excluded competition-specific content to reduce the risk of directly leaking answers.
However, the comparison does not include skill-building costs in the execution budget. Batch distillation costs about $40 per repository on average and uses GPT-5.5 and GPT-5.6 Sol. This makes the approach better suited to reusable team knowledge bases whose costs can be amortized over time than to one-off tasks. Engineering teams should next examine upstream version drift, skill supply-chain risks from malicious repositories, and whether third parties can reproduce the gains across different models and agent frameworks.