模型安全研究
Safety Neurons in Diffusion LLMs Transfer Across Architectures; Pruning Raises Refusal-Bypass Rate to 86.6%
New research suggests that diffusion LLMs converted from autoregressive models may also inherit safety mechanisms concentrated in a small number of neurons. The researchers also used an open diffusion model to generate adversarial prompts offline, but the promised full code has not yet been released.
A study included in arXiv’s latest batch on August 10 examines whether diffusion large language models truly have a safety structure distinct from that of autoregressive models. These models replace token-by-token generation with multiple rounds of parallel denoising. Because models such as Dream and Fast-dLLM are initialized from the Qwen2.5 family, the researchers asked whether safety alignment is also carried over unchanged during weight conversion.
In white-box experiments, the researchers first identified a small number of “safety neurons” by comparing activation differences between safe and harmful prompts, then tested the effects of pruning them. LLaDA’s attack success rate rose from 2.6% to 73.8%, while Dream’s increased from 1.9% to 86.6%. Transferring neuron locations identified in Qwen2.5 to Dream also raised the success rate to 73.2%, while Fast-dLLM’s increased from 7.0% to 86.3%. When selecting just 0.8% of neurons, the overlap in identified locations was 41.8% between Fast-dLLM and Qwen and 30.9% between Dream and Qwen, supporting the hypothesis that safety mechanisms are inherited across architectures and are excessively concentrated.
The study also introduces SN-Guided Diffusion. It uses LLaDA, whose internal activations can be inspected, as a surrogate model and steers the local denoising process away from activation regions that would trigger refusals. The resulting prompts are then transferred to black-box targets. The paper reports maximum success rates of 77.1%, 86.9%, and 74.3% against Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, and Gemini-2.5-Flash-Lite, respectively, requiring about 20 offline generations per query.
This does not mean diffusion models are inherently less safe. The results depend on the specific datasets, automated judges, model versions, and definitions of success used, and the study does not measure real-world harm. A larger reproducibility concern is that the GitHub repository currently contains only a README and still labels the code as “coming soon.” For model developers, the key questions are whether alignment signals can be distributed across more layers and features, and whether safety evaluations can incorporate adaptive attacks involving pruning, representation transfer, and surrogate-model generation instead of testing only surface-level refusal rates.