模型訓練與推理
DASH Dynamically Assigns Distillation Weights Based on Reasoning Trajectories, Outperforming Fixed-Weight Baselines Across Three Qwen3 Model Sizes
Rather than treating teacher–student discrepancies at every token equally, DASH adjusts how far supervision can propagate based on the entire generation trajectory. The authors report consistent gains on three mathematics benchmarks and have released the training code, evaluation pipeline, and LoRA weights.
Released on August 6, DASH attempts to address a temporal-structure problem in on-policy self-distillation (OPSD). These methods first have the student generate a reasoning trajectory, after which a teacher with access to privileged information, such as a reference answer, provides token distributions at every prefix the student actually traversed. Standard OPSD, however, aggregates the KL discrepancies across positions using nearly fixed coefficients, making it unable to distinguish cases where the same local deviation leads to different downstream outcomes.
DASH first clips the local forward-KL contribution of each vocabulary item, then compares the signal at each position with the sequence-wide average. This difference is passed through a sigmoid to produce a stop-gradient propagation gate. A backward recursive aggregation then determines the supervision horizon over which subsequent discrepancies should affect the current weight. Local discrepancies below the average open a longer horizon, while above-average discrepancies shorten propagation. Because the teacher and student distributions are already required outputs in OPSD, the authors say the method requires no additional model forward pass and adds less than 1% to step time.
The study evaluates Qwen3-1.7B, 4B, and 8B using Avg@12 on AIME 2024, AIME 2025, and HMMT 2025. Compared with paired reruns of standard OPSD, DASH raises the three-benchmark average scores from 41.87 to 45.07, from 63.60 to 65.00, and from 64.80 to 66.40, respectively. It leads in all nine model-size-by-benchmark combinations. The repository provides complete training and evaluation code, three LoRA adapters, pinned package versions, and hardware configurations. The main training runs reported in the paper used up to eight A800 80GB GPUs.
From an engineering perspective, the work shows that dense distillation signals do not have to be limited to token-by-token imitation: trajectory history can also be incorporated at low cost. The results are currently limited, however, to mathematical reasoning, Qwen3, and a single primary training seed. The README also explicitly notes that vLLM sampling and bf16 reduction are not bitwise deterministic. The next questions are whether the results reproduce across multiple seeds, whether coding and tool-using agents can benefit, and whether path-dependent weighting amplifies teacher bias when the privileged reference is incomplete or incorrect.