AI 評測與安全
MIST Finds All 23 Models Susceptible to Misleading Prompts; SCOPE Cuts Qwen3-4B’s Answer-Flip Rate from 35.0% to 16.3%
A new benchmark uses four paired contexts to distinguish whether models correctly evaluate evidence or simply ignore all external information. By changing only the configuration of DPO preference data, SCOPE reduces answer flips caused by misleading contexts while preserving accuracy in correct and irrelevant contexts.
After receiving retrieved documents, user opinions, or problem-solving hints, do language models actually evaluate the evidence—or merely follow text that appears authoritative? MIST-1000 rewrites the same set of 1,000 reasoning questions into four versions: no additional information, misleading hints, correct hints, and irrelevant information. The questions, answer choices, and ground-truth answers remain unchanged. The researchers also introduce the SC2W metric, which counts only cases where a model initially answers correctly but switches to an incorrect answer after a misleading signal is added. This avoids treating questions the model could not answer in the first place as evidence of contextual vulnerability.
All 23 API-based and open-weight models exhibited answer flips, with misleading signals reducing accuracy by an average of 17.1 percentage points. GPT-5.5 still recorded an SC2W of 10.5%, while Qwen3-4B reached 35.0%. The research team therefore proposes SCOPE. It first identifies trajectories in which the base model answers correctly without a hint but incorrectly with a misleading hint, then constructs pairs consisting of a complete correct response and a complete response that follows the misleading signal. The same pairs are distributed evenly across all four contexts and trained using standard sigmoid DPO and LoRA, without introducing a new optimization loss.
SCOPE reduces Qwen3-4B’s SC2W to 16.3% and Llama-3.2-3B’s from 31.5% to 20.6%. Neither model loses accuracy in the clean, correct-context, or irrelevant-context settings. By contrast, standard DPO trained only on misleading contexts lowers Llama-3.2-3B’s correct-context accuracy from 78.5% to 56.4%, exposing the side effect of indiscriminately distrusting context. The code, training data, and evaluation data have been released publicly.
This is still not deployment-level evidence of a defense against prompt injection. MIST is a controlled, text-only diagnostic; 800 of its questions were adapted from existing public benchmarks; and the main mitigation experiments cover only two small model families and a single decoding seed. Engineering teams should investigate whether the method extends to real-world RAG documents, tool outputs, and multi-turn agents, as well as whether reducing SC2W compromises the ability to incorporate new knowledge.