Ollama Preview Adds DFlash2, with MLX Speculative Decoding About 14% to 25% Faster Than Native MTP

Ollama released the v0.40.12-rc0 preview on October 27, bringing DFlash2 support merged the previous day to the MLX inference backend. For developers running local models on Apple chips, the update adds a decoding option alongside native multi-token prediction (MTP). At the time of review, the release was still marked as a pre-release. Release announcement

DFlash2 adds in-block dynamic convolution and a candidate selector to an architecture that generates drafts in parallel. The convolution lets adjacent positions exchange local information; the selector rescored candidates for the next position based on the previously selected token, giving the draft sequence coherent dependencies. The draft itself still requires only one parallel forward pass, while a smaller selector handles sequential selection. Architecture documentation

Ollama’s implementation proposes seven tokens per round, then verifies them using the existing snapshot rollback path. Draft sampling applies only the request’s temperature; top_k, top_p, min_p, and repetition penalties are handled by the target model during verification, and the candidate distribution is also passed to the verification process. The proposal also enables qwen3.5-series target models to accept DFlash drafts and checks configuration dimensions before loading weights. Merged proposal

Performance tests used NVFP4-quantized qwen3.8:27b to compare built-in MTP with DFlash2. Across M5 Max, M5 Pro, and M2 Ultra, and with prompt lengths of 2K and 32K, maintainers reported generation speedups ranging from 1.14× to 1.25×. For example, the M5 Max 2K test increased from 57.0 to 71.0 tokens per second. The tests enabled thinking and used temperature 1, top_p 0.95, and top_k 20; the medians were calculated from 16 and 12 requests for the two prompt lengths, respectively. Test results

These results support further evaluation, but remain a limited sample provided by the maintainers. Engineering teams should next test with real Chinese prompts, long responses, and different sampling settings, measuring acceptance length, generation speed, and additional memory costs, and wait for a stable release. Inference support should also be distinguished from training reproducibility: the Speculators documentation states that the public DFlash2 inference implementation does not disclose its training objective, and its training recipe remains experimental. Training limitations