AI 安全與代理框架
SHE Rewrites the Safety Harness from Agent Execution Traces, Cutting Attack Success Rate from 17.1% to 5.5%
SHE decomposes an agent safety harness into a system prompt, rule library, safety memory, and tool policy, then locally revises them based on failure traces. On Agent-SafetyBench, it reduces both attack success and false refusal rates, though real-world deployments must still guard against evaluator bias and unchecked rule growth.
Shanghai AI Laboratory, Fudan University, and other institutions have introduced Safety Harness Evolution (SHE), which transforms agent safety from a fixed system prompt into a non-parametric layer that can be continuously revised using historical execution traces. The framework keeps the underlying model and task environment unchanged while decomposing the harness into four artifacts with clearly defined responsibilities: the system prompt establishes the overall trust boundary; the rule library classifies risks and determines whether to allow, warn, block, or sanitize; safety memory stores recurring cases that cannot be repaired; and the tool policy constrains invocation permissions and recovery procedures after an action is blocked.
In each evolution round, the system first replays clean and adversarial tasks, produces structured diagnoses by harm domain, attack surface, and failure mode, and then routes each issue to the smallest responsible artifact. Candidate modifications must pass formatting and plausibility checks, and replace the current version only when they improve the safety score without reducing utility on normal tasks. Rejected modifications and the reasons for rejection are retained as references for the next round. This design aims to prevent agents from merely expanding prohibition rules to game safety scores.
The researchers evolved the harness using 15 tasks from Agent-SafetyBench, then tested it on the remaining 185 task IDs with clean inputs, context poisoning, indirect prompt injection, tool tampering, memory injection, and composite attacks. After evolution, SHE achieved an average attack success rate of 5.5%, compared with 17.1% for the static SafeHarness; usability accuracy increased from 31.6% to 47.6%. Comparing SHE before and after evolution alone, the attack success rate fell from 8.6% to 5.5%, while the improper blocking rate for clean requests decreased from 25.7% to 19.8%. On AgentHarm, which was not used during evolution, the Harm Score dropped from 19.8% to 9.8%.
The code, fixed data splits, and harness seeds have been released under the Apache-2.0 license, but the generated harnesses, complete rollouts, credentials, and external benchmark packages are not included in the repository. Engineering teams can therefore inspect the evolution process, but must still cover their own model inference and evaluation costs to reproduce the main results. Future work should examine whether long-term updates create rule conflicts or evaluator overfitting, and whether automatically generated tool policies can support human audits and rollbacks in real-world authorization systems.