AI research
One-Shot OPD Captures Most Full-Data Distillation Gains With a Single Query, Shifting the Bottleneck to Teacher-Signal Absorption
Researchers found that repeatedly sampling from a single query while giving the student model token-level teacher supervision can, after 300 steps, cover 71.5% of the state reached by full-data training. Adding 16 semantically diverse queries raises coverage to 98.9%, though the findings remain limited to the specified small models, teachers, and benchmarks.
Researchers from Tsinghua University and other institutions have released One-Shot OPD, a study examining whether on-policy distillation for large language models truly requires a large number of diverse prompts. In OPD, the student model first generates its own rollouts, after which a frozen teacher provides dense distributional supervision at every token state the student actually visits. As a result, “one query” does not mean one fixed training example: each step includes 64 stochastic rollouts, and every token prefix creates a new supervised state.
In the mathematics setting, training on a single query for 300 steps produced an average score of 68.5 across MATH-500, AMC 2023, and AIME 2025, compared with 69.8 for the full dataset of 17,000 queries. The single-query setup captured 87% of the total OPD gain and covered 71.5% of the state reached by full-data training. The effect also appeared in code generation, instruction following, and agentic tool use, recovering 73%, 66%, and 64% of the teacher–student gap, respectively. The researchers then used BGE-M3 semantic clustering to select diverse queries. Sixteen queries per domain achieved 98.9% state coverage and matched full-data training under multi-teacher OPD.
The paper describes the bottleneck as “data-rich, algorithm-starved”: whether training used 1, 4, 16, or all available queries, the fraction of the remaining teacher–student distribution gap eliminated by each update declined over the course of training. Even when the original 64 trajectories were reused without change, alignment still required hundreds of steps, suggesting that the slowdown is not merely caused by waiting for new states to appear.
This does not mean task data is no longer useful. The experiments depend on particular teachers, model families, and evaluations, while OPD’s capability ceiling is also constrained by the teacher distribution. The publicly released Apache-2.0 code is based on veRL. Reproducing the reference configuration requires a single machine with eight 80GB H100 or A100 GPUs, and the data must be reconstructed from public sources. Future work should test larger models, teachers of varying quality, and whether reducing the data creates capability gaps that existing benchmarks fail to capture.