模型訓練
DASH Dynamically Allocates Distillation Weights Based on Reasoning Divergence, Raising Qwen3-1.7B’s Math Average by 3.2 Points
Rather than treating teacher–student differences at every token equally, DASH adjusts the scope of supervision based on how divergence changes across an entire reasoning sequence. All three Qwen3 sizes outperform OPSD under the same conditions, but selecting the best checkpoint by average test-benchmark score may overestimate generalization gains.
Verifiable-reward reinforcement learning typically provides a success-or-failure signal only after an answer is completed. On-policy self-distillation (OPSD), by contrast, has the student model generate reasoning trajectories and then uses the same model—after additionally receiving a reference answer—as the teacher, providing token-level distributions at every prefix actually traversed by the student. The problem is that standard OPSD averages the KL divergence at each position with equal weight, without distinguishing whether divergence is growing, converging, or merely fluctuating temporarily.
The [DASH paper](https://arxiv.org/abs/2608.06243) compares the divergence at each position with the sequence mean, transforms it through a sigmoid into a propagation gate, and then recursively aggregates it backward from the end of the sequence. Below-average divergence opens a longer supervision range, while high-divergence positions shorten propagation. The gate uses stop-gradient, so it only reallocates existing distillation gradients without adding another teacher or student forward pass. The [public implementation](https://github.com/DBtxy/DASH-OPSD) reports less than 1% additional step time and provides LoRA adapters for Qwen3-1.7B, 4B, and 8B.
The team evaluated the models on AIME 2024, AIME 2025, and HMMT 2025 using Avg@12. Compared with their own rerun of OPSD, the macro-average across the three benchmarks rose from 41.87 to 45.07, from 63.60 to 65.00, and from 64.80 to 66.40 for the three model sizes, respectively. Scores were higher across all nine model–benchmark combinations. DASH’s engineering appeal is that it can be layered onto an OPSD pipeline that already computes the teacher’s full vocabulary distribution, requiring no additional inference, although that prerequisite remains fairly expensive.
The results also have clear limitations: the experiments cover only Qwen3 and math problems. Each method was trained for 200 steps, and the best checkpoint was selected directly using the average score across the three reported benchmarks rather than an independent validation set. Future work should examine whether the gains persist at a fixed training endpoint and on code tasks or long agentic trajectories, as well as whether dynamic weighting amplifies teacher bias when reference answers are incorrect.