推論系統
Liquid AI Adds DSpark Draft Models to LFM2.5, Delivering Up to 3.18× Faster Decoding on a Single H100
Liquid AI has released three DSpark draft models with roughly 300 million parameters, enabling LFM2.5 to verify multiple candidate tokens at once through speculative decoding. Tests with batch size 1 and greedy decoding showed speedups of up to 3.18×, but SGLang support remains in an open PR, leaving deployment maturity yet to be validated.
On August 20, Liquid AI released DSpark draft models for LFM2.5-1.2B-Instruct, 2.6B, and 8B-A1B, along with Safetensors and GGUF weights and integrations for llama.cpp and SGLang. This does not quantize or modify the target model. Instead, during decoding, a lightweight model with approximately 296 million to 328 million parameters first proposes a sequence of candidate tokens, which the target model then verifies in a batch with a single forward pass.
DSpark’s drafter combines a parallel backbone, sequential Markov heads that model dependencies between adjacent tokens, and a verifier that prunes low-confidence suffixes based on survival probability. This addresses a common problem with purely parallel drafters: as candidate sequences grow longer, later tokens are increasingly likely to be rejected because they lack dependencies on preceding context, causing failed candidates to waste computation.
Liquid AI tested the models with batch size 1 and temperature 0. On MATH500 using an H100 80GB, 8B-A1B throughput increased from 428 to 1,362 tokens/s, a 3.18× speedup. On an M4 Max, however, its average speedup was only 1.18×. The team attributed this to llama.cpp’s MoE implementation for Metal and to multi-token verification activating more experts. Across five benchmarks, the 2.6B model achieved average speedups of 2.67× on the H100 and 2.27× on the M4 Max, while average latency in multi-tool scenarios fell by 57%.
With greedy decoding, candidates are accepted only when they match the target model’s distribution; when a candidate is rejected, the target model’s token is used instead. The output can therefore remain identical to the baseline. However, this “no quality degradation” conclusion cannot be directly generalized to temperature-based sampling, long contexts, concurrent multi-user workloads, or memory-constrained serving. At the time of verification, the PR adding LFM2 support to SGLang had not yet been merged. Engineering teams should first retest accepted sequence lengths, peak memory usage, and concurrent throughput on their own prompt distributions before deciding whether to deploy DSpark in production.