推論與基礎設施
vLLM 0.26 Expands Cross-Layer KV Caching and Hybrid Attention, Accelerates DeepSeek-V4 Inference Kernels
vLLM 0.26 allows each KV-cache group to use its own attention backend and adds object storage to its tiered cache. The release also focuses on optimizing DeepSeek-V4, ROCm, XPU, and speculative decoding, though its official microbenchmarks do not directly represent real-world serving throughput.
vLLM released version 0.26.0 on July 27, incorporating 411 commits from 212 contributors. What matters most to deployment teams is not the number of newly supported models, but that the engine now treats hybrid architectures and large clusters as the norm: each KV-cache group can select a different attention backend, while sliding-window attention is now an explicitly declared backend capability. This means models combining full attention, local attention, or other state-space modules no longer have to be forced through a single execution path.
KV offloading has also expanded beyond simply moving data to the CPU, evolving into tiered secondary storage. The new release adds metrics such as read and write volume and lookup latency, supports access to object storage through workload identity, and recognizes data-parallel replicas. The encoder cache also gains a CPU connector. These components can benefit long-context workloads and prefix reuse, but the actual gains will still depend on remote-storage latency, network traffic, and cache hit rates—not just reductions in GPU memory usage.
On the hardware side, the main theme is cross-vendor optimization. The project adds a dedicated routing kernel for DeepSeek-V4, with an official commit reporting a 2.94% improvement in end-to-end time per token; the standalone `fused_topk_bias` kernel is 1.5× to 2× faster. ROCm gains a two-stage compressor, sparse prefill/decode, and DSpark speculative decoding, while XPU also adds DSpark support. Qwen3.5 receives additional fusion optimizations for RMSNorm, communication, and H20 execution paths. Most of these figures come from controlled environments documented in individual pull requests, and the release notes do not provide an overall comparison using standardized hardware, batch sizes, and sequence lengths.
In addition, the Rust frontend now supports audio, video, and native `vllm-bench`, while the API adds an endpoint plugin framework. Before upgrading, teams should rerun their own stress tests for long-context workloads, hybrid models, and multi-replica deployments. In particular, they should monitor KV-tier lookup latency, speculative-decoding acceptance rates, and whether the new kernels maintain consistent numerical results across CUDA, ROCm, and XPU.