推論系統
Uno Uses Discrete Diffusion to Assist Autoregressive Models, Enabling Parallel Decoding Without a Separate Draft Model
Uno attaches lightweight diffusion weights to an autoregressive LLM, then uses Ψ-Spec to propose multiple tokens at once while preserving the original model’s sampling distribution. The authors report speedups of up to 3×, though the current figures are limited to the research team’s specific models, hardware, and inference engine.
Autoregressive language models must generate tokens one at a time. Although speculative decoding can reduce the number of sequential steps performed by a large model, it usually requires training or deploying a separate draft model. Uno, proposed by researchers including the IFM team, instead adopts a “diffusion-assisted” architecture: the existing AR weights continue to define the probability distribution through next-token prediction, while a separate, lighter set of diffusion weights learns to propose a sequence of candidate tokens simultaneously.
The Ψ-Spec sampler is what ultimately determines whether the result is distributionally equivalent. It assigns candidate generation to the diffusion path, after which the AR path verifies or rejects those candidates. Thus, the paper’s use of “lossless” means that the final samples still follow the base AR model’s distribution—not that every run is guaranteed to produce identical text, nor that the floating-point computations are exactly identical. The system provides a linear sampler optimized for aggregate throughput and a tree sampler designed to improve single-request latency.
In the tested configurations, the authors report speedups of up to 3× over base AR decoding, with throughput exceeding that of the speculative decoding methods used for comparison at every batch size. The 8B version also outperformed the authors’ selected DiffusionGemma 26B and Mercury 2 baselines on agentic tool-use, coding, and long-context evaluations. These accuracy comparisons do not establish comprehensive superiority under equal-cost or general-purpose serving conditions.
The Apache-2.0 repository includes a Nano-vLLM-derived inference engine, training and evaluation pipelines, and recipes and public checkpoints for K2 Horizon 0.9B and 7B, as well as Qwen3 8B. The Qwen3 implementation adds diffusion capabilities through conditional LoRA, indicating that existing open-weight models may not need to be pretrained from scratch.
Engineering teams should next examine Ψ-Spec’s acceptance rate, KV cache cost, and tail latency across different GPUs, long outputs, high-concurrency workloads, and quantized models. The current installation process is pinned to specific combinations of PyTorch, CUDA, and FlashAttention, and direct integration into production deployments based on mainstream vLLM or SGLang has yet to be demonstrated.