AI 安全／模型對齊
Boundary-Aware Self-Distillation Narrows the Scope of LLM False Refusals, but Safety Gains Still Trade Off Against Over-Refusal
A new study models safety policies as fine-grained refusal boundaries within topics and uses paired prompts and compensatory data to prevent models from blocking entire sensitive topics. Experiments on Qwen3-8B show that the method can substantially reduce false refusals near the boundary, but the findings remain limited to political persuasion, a single primary model, and a data pipeline that has not been fully disclosed.
The Multiverse Computing team reframes safety alignment for large language models as a “narrow-boundary” problem: deployers generally do not want models to refuse entire topics such as politics, cybersecurity, or medicine, but only intentions within those topics that violate policy. For example, a model should answer questions about electoral systems but should not write manipulative messages targeting a specific demographic group. A guard model based solely on topic classification struggles to represent this distinction.
Using Qwen3-8B and political persuasion as its primary test setting, the study builds an offline self-generation pipeline. The model first generates refusal trajectories, which are then validated by a safety model. If a single steering attempt fails to produce a valid refusal, the system addresses gaps in data coverage through progressively escalated retries, transplantation, or length control. A single generation pass misses 19.88% of audited prompts; escalated retries reduce that gap to 0.20%. However, adding only this refusal data can also push the safety boundary outward.
This side effect is pronounced. After training on refusal data completed through Escalate, the refusal rate for targeted political prompts rises from 9.47% to 84.75%, while the average unsafe-response rate across three broad harm evaluations falls from 26.26% to 0.14%. At the same time, over-refusal on safe prompts measured by XSTest rises from 2% to 74%. The researchers therefore add compliant responses generated and validated by the target model itself, along with harmful/benign prompt pairs that share topical anchors and differ only in intent.
In one paired ablation, boundary data reduces the false-refusal rate on the benign side from 32.94% to 4.16%, while the refusal rate on the harmful side declines slightly from 91.88% to 87.72%. This suggests that safety teams cannot track jailbreak success rates alone; data coverage, the benign compliance rate, and recall and precision on both sides of the boundary should all be treated as model deployment metrics.
However, the paper’s main results come from a single topic, Qwen3-8B, and a fixed LoRA configuration. Data overlap is incomplete in some comparisons, and the model revision was not pinned. The dataset and generation code are expected to be released later under terms restricting their use to safety research, while scoring coverage from WildGuard and LlamaGuard is also incomplete. It therefore cannot yet be assumed that the method will generalize directly to Chinese-language contexts or other risk policies.