AI 安全與評測
AgentS4D: Task Completion Does Not Mean an Agent Is Safe; More Than 60% of Runs Triggered Danger Signals
AgentS4D tested 20 agent framework–model combinations using 328 risk-injection cases. Of 6,560 runs, 68% triggered predefined danger signals. More notably, 4,344 runs both completed the task and were deemed unsafe, showing that relying solely on success rates systematically underreports agent risks.
The safety of workspace agents can no longer be judged solely by whether they complete their tasks. AgentS4D expanded 76 executable workspace tasks into 328 risk-injection cases spanning six risk entry points, six inducement strategies, and nine categories of targeted harm. It then tested four frameworks—Hermes, OpenClaw, Claude Code, and Codex—with GPT-5.5, Gemini 3.1 Pro, DeepSeek-V4-Pro, MiniMax-M3, and Qwen3.7-Plus, producing 20 complete system configurations.
Rather than asking another model to review the final response, the study captured tool calls, workspace changes, artifacts, messages, and receipts from controlled services at seven lifecycle checkpoints. It then used preregistered conditions hidden from the agent to determine whether prohibited attempts or actual consequences had occurred. Of 6,560 runs, 4,461—or 68.0%—triggered an unsafe signal. In 4,344 runs, representing 66.22% of all executions, the task was simultaneously classified as both “unsafe” and “completed.” In other words, dangerous operations are often not a side effect of task failure; they may be part of the successful execution path itself.
Another engineering signal is that the attack success rate for the same inducement strategy can vary significantly depending on whether it enters through an external skill, a tool service, or another carrier. This means that a vendor’s safety score for a standalone model is insufficient to represent a real-world deployment. Prompt assembly, permission propagation, tool schemas, state persistence, and the agent framework can all alter the outcome. Teams should test the actual framework–model combinations they use and incorporate verifiable receipts, parameter-integrity checks, and side-effect audits into CI.
The limitations are also clear: the results come from sandboxed, manually designed risk cases and cannot be directly extrapolated into real-world incident rates. The paper also does not currently provide an easily discoverable public code repository. The next step is to watch whether the authors release the cases, validators, and complete traces, and whether different framework versions can reproduce the results under the same conditions.