LLM 推論系統
xPress Restores Causal Dependencies to Diffusion-Based Drafts, Boosting Qwen3-8B Decoding Throughput by About 1.3× on Average
xPress adds a single parallel causal refinement step between block-diffusion drafts and the target LLM, reducing early rejections caused by drafts in which individual tokens are plausible but the sequence as a whole is not. The study reports an average increase of about 30% in accepted draft length, but no public implementation is currently available for independent reproduction.
Speculative decoding first uses a smaller drafter to propose a batch of tokens, which the target model then verifies in a single pass. As long as the draft distribution is correct, this can accelerate decoding without changing the target model’s output. Block-diffusion methods such as DFlash go a step further by generating an entire draft span in a single forward pass, but their final denoising step samples each position independently. Although each token may individually have a high probability, their combination may not conform to the conditional distribution of the autoregressive target model, causing verification to reject the remainder of the draft within the first few positions.
Released on August 3, xPress adds a lightweight causal refiner after the diffusion drafter. Instead of rerunning an autoregressive loop token by token, it processes the entire draft block in parallel, propagates dependencies from earlier positions to later ones, and then sends the refined sequence to the target model for verification. In other words, the new module attempts to restore the joint-distribution structure lost during the final sampling stage of block diffusion while preserving the main speed advantage of parallel draft generation.
The authors evaluated Qwen3-8B on seven benchmarks covering mathematics, coding, dialogue, and other tasks. Compared with the original DFlash, they report an average increase of about 30% in accepted draft length, with a maximum of 56%. End-to-end decoding throughput averaged about 1.3× the DFlash baseline and reached as high as 1.7×. These results also show that improving the draft model’s local speed alone is insufficient: actual system-level gains depend on how many consecutive tokens the target model can accept and whether the cost of the added refinement layer can be amortized through fewer verification rounds.
At this stage, the results cover only a single 8B target model, and the paper’s webpage does not link to xPress code or weights. It is therefore not yet possible to assess the gains under different batch sizes, long-context workloads, quantized models, or concurrent serving conditions. Engineering teams should pay particular attention to forthcoming integration data for SGLang, vLLM, or llama.cpp. If the causal refiner must be retrained for every target model, deployment and model-version management costs could also offset some of the acceleration benefits.