代理安全
StepGuard Reviews Agent Actions Before Tool Execution, but Adds About 2.53 Seconds of Inference per Task
A team including the Shanghai AI Laboratory has released StepGuard, a 4B-parameter model that uses paired safe/dangerous trajectories and Balance-GRPO to calibrate overblocking and missed detections. It substantially reduces attack success rates in two agent environments, but its training corpus and data generator have not yet been released, and it still misses some policy violations.
Most agent guard models inspect prompts, responses, or entire completed trajectories. If a dangerous step involves sending a file, modifying a network policy, or executing a transaction, a post hoc judgment cannot reverse the tool’s side effects. StepGuard moves the decision point to before each tool call: it takes the current trajectory and proposed action as input, then outputs a safety label, risk type, and dangerous step. The same 4B model can also be used for offline trajectory auditing.
Its StepGen pipeline first constructs safe and dangerous trajectories that share the same prefix and diverge only at the critical action. It also adds visually similar but legitimate tool-reuse cases to prevent the model from blocking an action merely because it sees a particular tool name. After cold-start supervised fine-tuning, Balance-GRPO reweights the advantage according to the accuracy gap between safe and dangerous samples in each batch of rollouts, shifting training resources toward whichever class is currently weaker. Ablation experiments show that, compared with standard GRPO, overall accuracy/F1 rises from 81.5/81.9 to 82.2/82.1, while the accuracy gap between the two classes falls from 13.0 to 8.0.
When evaluated with the same Qwen3.6-35B-A3B agent, StepGuard achieves an attack success rate of 1.2 and utility of 90.7 on AgentDojo; on AgentDyn, the corresponding figures are 9.3 and 66.7. The team summarizes this as an average 77.3% relative reduction in attack success rate versus an unprotected configuration, with utility declining by 2.8 percentage points. This protection is not free: when every call is inspected, each safeguard check takes an average of 599.9 milliseconds and generates 195.5 tokens. Each task is checked an average of 4.22 times, totaling about 2.53 seconds and 825 tokens.
The model weights and evaluation code have been released, but the 7,000-example training set and StepGen have yet to be published, so outside researchers currently cannot fully reconstruct the data quality or teacher bias. The authors’ analysis of 575 errors also shows that StepGuard is better at detecting tool-mediated technical harm, but may miss institutional or policy violations such as submitting coursework under someone else’s identity. Engineering teams should treat it as an additional risk classifier, not as a security boundary that replaces permission isolation, human approval, and rollback-capable tool design.