AI 安全與評測
RAG-Safety-Bench Isolates Retrieval Quality as a Variable, Showing Safety Fine-Tuning May Not Withstand Answer-Bearing Documents
The new benchmark holds retrieved content constant and uses four conditions to measure how RAG changes five open models’ responses to harmful questions. Documents containing answers made every model more likely to produce accurate but dangerous content, while safety judges showed only moderate agreement.
A research team at the University of Ottawa has released [RAG-Safety-Bench](https://arxiv.org/abs/2609.11758), seeking to answer a question often overlooked in deployment: after a base model passes safety evaluations, do its guardrails remain effective when it is connected to enterprise documents or other knowledge bases? Rather than performing vector retrieval, the benchmark directly supplies fixed documents, eliminating misleading safety gains caused by retrieval quality.
The benchmark evaluates the same set of harmful questions under four conditions: no RAG, an oracle document containing the answer, a document on the same topic that does not contain the answer, and a safe but irrelevant random document. The full version covers 987 questions, while the paper primarily analyzes a balanced subset of 346 questions, capped at 20 questions per subcategory. Harm categories include violence, fraud, cyberattacks, weapons, and self-harm. The models tested were Gemma 3 12B, Llama 3.1 8B, Ministral 3 8B, Qwen 2.5 7B, and Phi-4 14B.
The results show that oracle documents increased every model’s accuracy in answering harmful questions. For example, Ministral 3 8B rose from 26.0% without RAG to 90.7%, while Qwen 2.5 7B increased from 15.9% to 82.9%. However, safe documents on the same topic but lacking the answer worsened safety for only some models. Irrelevant long-form text was generally the safest condition, indicating that the issue cannot be reduced to “longer context is more dangerous.” From an engineering perspective, this means that document trust levels cannot replace post-generation safety checks, and RAG evaluations should cover answer-bearing, topically related, and control documents.
The study determined harmfulness by majority vote among LlamaGuard, ShieldGemma, and WildGuard, but the four-way Fleiss’s kappa was only 0.56. More importantly, although the [public repository](https://github.com/The-Safe-AI-Lab/RAG-Safety-Bench) provides the data, configurations, and local scoring code, it does not include the raw model responses, judge traces, or artifacts from the private v2 re-evaluation used in the paper. The study also did not test commercial models, so it is currently better suited to designing internal red-team matrices than to claiming that specific products have been comprehensively compared.