AI 代理與開發工具
AutoDesign Lets Agents Rewrite Their Own Design Harness, Raising Average Poster Scores by 12.4 Points Across Seven Configurations
AutoDesign uses an outer agent to analyze execution traces and modify the production system, with an independent development set determining whether updates are accepted. The open-source DesignHarness outperformed several commercial systems on the authors’ PosterBench, but automated scores had a correlation coefficient of only 0.34 with human preferences.
AutoDesign elevates the goal from “improving a single output” to “improving the entire harness that repeatedly produces outputs.” The inner Designer reads papers, creates editable posters in HTML, and makes targeted revisions based on rule-based checks and visual-model critiques. The outer MetaHarnessOptimizer aggregates traces and scores from multiple rollouts, then assigns coding agents to modify one of five component categories: context, tool specifications, runtime, orchestration, or evaluation feedback. Only one category may be changed per iteration. A candidate version becomes the next version only if it improves training-set performance without reducing performance on a development set hidden from the modification agent.
The team says the seven-day evolution process used 224 subagents, completed at least 123 recursive iterations, and accepted 54 updates. In a controlled test using the same ten papers, connecting DesignHarness to seven model-and-coding-agent configurations raised the average PosterBench score from 54.99 to 67.39. On the full 100-paper main leaderboard, the Claude Code/Claude 4.8 combination scored 78.32, outperforming Claude Design with the same model by 7.45 points. A complete generation run can involve 253 tool calls and 11 editing rounds, taking about 40 minutes.
The engineering value of this work extends beyond automated poster creation. It demonstrates how agents’ failure traces can be converted into versioned, reversible system modifications without updating model weights. The [repository](https://github.com/Yaxin9Luo/AutoDesign) also makes DesignHarness and the evaluation pipeline publicly available. However, PosterBench uses both rules and VLM judges, and its correlation coefficient with human preferences in blinded evaluations is only 0.34. Applications beyond posters—including slides, webpages, and videos—also remain demonstrations. Engineering teams should next focus on evaluator reward hacking, development-set design across data distributions, and whether long-term self-modification could accumulate security vulnerabilities.