AI 安全
Self-Evolving Agents Can Turn One Unsafe Success Into a Permanent Skill; Three Malicious Tasks Raise the Cross-Session Attack Rate to 35.3%
SkillMisevo-Bench tracks how agents generate, retrieve, and reuse skills from execution traces, finding that all 21 evolution-capable configurations produced unsafe artifacts. SafeEvolve can reduce downstream harm, but the evidence is currently limited to four CLI agents and controlled computer-use tasks.
The risks posed by agent “memory” extend beyond storing sensitive text: an agent may also distill a successful but dangerous action into a reusable procedure. The newly released SkillMisevo-Gym breaks this lifecycle into skill creation, retrieval, execution, and cross-session persistence. After each task, it resets the conversation, working directory, and process state; only the versioned skill repository persists across tasks. An attacker can submit malicious tasks at a limited number of positions in the learning history but cannot directly modify skill files, the retriever, or subsequent tests.
SkillMisevo-Bench compares four CLI harnesses—Claude Code, Codex, OpenClaw, and Hermes—and six skill-evolution methods. Across 625 primary experiments, it holds the execution model constant in an effort to attribute differences to the method and harness. All 21 evolution-capable configurations produced unsafe skills, and 15 caused actual harm in freshly initialized, clean sessions. Adding just three malicious tasks increased the carryover attack success rate from 16.0% with no exposure to 35.3%. Under the full exposure budget, it reached 41.3%, while interleaving benign updates did not reliably remove the contamination.
The authors propose SafeEvolve, which uses a critic to identify and remove dangerous content before skills are written, incorporates provenance risk during retrieval, and prunes skills based on their history of harmful reuse. On AutoSkill and EvoSkill, it reduced unsafe retrieval and harm in clean sessions by 26.7 and 17.3 percentage points, respectively, while changing benign utility by an average of only 0.4 points. The engineering takeaway is straightforward: skill files should be treated as executable software supply-chain artifacts, with provenance, approval, versioning, and revocation mechanisms. They should not be trusted automatically simply because a task succeeded. One limitation is that the evaluation focuses on skill repositories and computer-use tasks. Whether long-term memory, model-weight updates, multimodal agents, and organic traffic exhibit the same accumulation curve remains to be verified.