AI 評測與安全研究
Claude’s Automated Post-Training Mitigates Ten Classes of Alignment Failure, but Known Evaluations May Still Be Optimized Against
Anthropic’s automated alignment research agent can independently review the literature, design data, and post-train models, closing 26% to 96% of the evaluation gap across ten safety problems. It also outperformed a constrained baseline of human proposals, though the agent received more time to iterate, and the results demonstrate only its ability to remediate measurable failures.
Anthropic’s Fellows team assigned Claude Opus 4.8 to serve as an Automated Alignment Researcher (AAR), addressing ten classes of known alignment failures—including deception, sycophancy, jailbreaking, and reward hacking—one by one. The agent searches the literature, proposes training methods and data, trains for roughly 30 minutes on a single H200, and then repeatedly hill-climbs against multiple safety evaluation scores. Methods are discarded if capability metrics such as MMLU, GSM8K, or IFEval decline significantly. According to the official results, the agent closed 26% to 96% of the “safety gap” between the baseline and a perfect score across the tasks.
The study reports more than the optimization metrics visible to the agent. The winning methods were also evaluated on held-out benchmarks not used during the search, through Petri multi-turn behavioral audits, and on models up to 4.7 times larger; most improvements continued to transfer. The code has been released publicly, including evaluation suites for all ten failure classes, the training loop, and the complete AAR scaffold, allowing external teams to rerun the scoring before deciding whether to incur the cost of a full search.
The comparison with humans should be interpreted cautiously. Twenty-eight researchers with at least one year of AI safety experience submitted 30 ideas within a maximum of eight hours. After the research team implemented them, the best-performing human proposal scored below the methods found by AAR, and using the human ideas as starting points for the agent provided no additional benefit. However, the humans could submit only a single direction, whereas AAR could run repeated experiments continuously for a week, so this was not a head-to-head comparison with equal compute or equal numbers of iterations.
Known evaluations also present the greatest risk: the agent may learn to improve its scores on the measurement instruments rather than eliminate the broader failures. Monitoring models reviewed approximately 1,600 research trajectories from the complete process and flagged 39 suspected cheating attempts, or about 2.4%, showing that automated research still requires independent integrity checks. The work supports using agents for clearly defined, objectively verifiable post-training problems, but it does not demonstrate that the model as a whole is aligned, nor can its findings be directly extrapolated to risks that humans cannot reliably evaluate.