推論系統
DARTree Builds Trees in Batches Before Pruning, Accelerating Qwen3-4B Decoding to as Much as 9.73× the Speed of Local AR
DARTree extends causal correction for diffusion draft models from a single path to multiple branches and replaces node-by-node heap search with batched expansion by depth. In tests on a single RTX 6000 Ada, it delivered higher average speedups across seven tasks than DFlash, DDTree, and Domino, but the code has not yet been released.
DARTree addresses a trade-off in diffusion-based speculative decoding: DFlash-style draft models can predict an entire 16-token block at once, making them fast, but they do not adjust later distributions based on tokens already selected within the block. Domino restores causal conditioning by adding an autoregressive correction head, but if best-first search is performed node by node, the correction head falls back to sequential execution.
The new method processes all active branches in batches, one tree depth at a time. It first uses a block drafter to produce shared representations and retains the top 64 candidates at each position. The existing Domino correction head then rescores them according to each branch prefix. A fixed width is retained at every depth, forming a supertree larger than the final verification tree. Only after construction is complete does the system select the 64 highest-scoring nodes in one pass using cumulative log-probability and a depth penalty. When the depth reward is non-positive, the authors prove that this node set remains prefix-closed and is equivalent to performing heap selection item by item on the materialized tree. It can therefore be passed to the target model for verification in a single tree-attention operation.
The authors evaluated Qwen3-4B and Qwen3-8B at temperatures 0 and 1 across four configurations on GSM8K, MATH-500, AIME 2025, HumanEval, MBPP, MT-Bench, and Alpaca. With Qwen3-4B at temperature 0, DARTree achieved an overall average speedup of 6.99×, compared with 4.58× for DFlash and 5.17× for Domino. On GSM8K, it accepted 12.97 tokens per iteration and reached a 9.73× speedup. The results preserve the target model’s output distribution, but not every workload approaches the peak: gains were smaller on chat tasks, and one AIME configuration was slightly slower than DDTree. All latency measurements used a batch size of 1, generation capped at 2,048 tokens, and a single RTX 6000 Ada. The evaluation does not yet cover high concurrency, long prompts, or data-center-scale serving. The DARTree implementation also remains unavailable, so engineering teams must wait for a reproducible core integration and data on memory costs.