AI 代理與自動化研究
Global Debugging Memory Helps AutoResearch Agents Avoid Repeating Mistakes, Raising AIDE Gold-Medal Runs from 22 to 38
With GPT‑5‑mini fixed as the model and a two-hour budget, researchers enabled a debugging advisor to share execution-environment constraints across search branches, allowing all 90 AIDE runs to produce valid submissions. The results suggest that agent scaffolds still contain substantial recoverable compute, though the experiments covered only nine tabular prediction competitions.
Automated research agents may not lack model capability; their compute may instead be wasted repeatedly fixing the same errors. A [study](https://arxiv.org/abs/2608.10424) released on August 11 examined the search processes of AIDE, ML‑Master, and MLEvolve and found that different code branches were isolated from one another: even after one branch had identified a package version, column format, or execution-environment constraint, other branches would still encounter the same failure again.
The team introduced a “global debugging advisor” that writes confirmed runtime constraints and lessons from fixes into cross-branch memory, then injects relevant information before the agent generates its next piece of code. Across ten seeds for each of nine MLE‑bench/Kaggle tabular tasks, AIDE’s valid submissions rose from 73/90 to 90/90, while gold-medal runs—those reaching the top 10% of the human leaderboard—increased from 22 to 38. ML‑Master’s gold-medal count also rose from 18 to 29. In AIDE’s search logs, the share of repeated errors fell from 46% to 7.8%, while the proportion of nodes that executed successfully increased from 54.7% to 79.0%.
The study also tested explicit instructions to perform hyperparameter tuning, as well as replacing a more rigid node-selection method with Thompson sampling and backtracking. In another experiment, deliberately planted errors in exploratory data analysis conclusions often did not alter subsequent model selection, suggesting that agents may merely be imitating the analysis workflow at a superficial level.
The implication for AutoML and research-agent engineering is that error memory, search policy, and budget control should be treated as first-class system components. The [public code](https://github.com/tingtang2/autoresearch-compute-recovery/tree/main) includes commands for treatment and control groups, but the maintainers note that parts of the workflow have undergone only syntax and wiring validation and have not yet been rerun end to end from that checkout. All major experiments also used only GPT‑5‑mini, tabular data, and fixed CPU time limits, so the findings cannot be directly generalized to deep-learning training, scientific simulation, or more capable models.