推論系統
Sol-Attn Dynamically Sparsifies Attention Within Online Softmax, More Than Doubling End-to-End Video Generation Speed
NVIDIA researchers have unveiled Sol-Attn, a training-free technique that selects key blocks during attention computation and approximates the contributions of skipped blocks. In NVIDIA's tests across several video models, it delivered roughly 1.9× to 2.34× end-to-end speedups, although its quality and generalization across hardware still require independent verification.
NVIDIA researchers have unveiled Sol-Attn, an attempt to reduce the attention cost of diffusion Transformers during high-resolution video generation. Existing dynamic sparse attention methods typically build a proxy score map before selecting key-value blocks based on either a fixed ratio or cumulative probability. The former cannot adjust compute according to query difficulty, while the latter can produce workload imbalances; both also incur routing and intermediate-tensor overhead.
Sol-Attn combines routing, sparse computation, and error correction into a single online softmax pass. Its core approach compares proxy scores against a threshold block by block on-chip, without first materializing a complete score map. Blocks below the threshold are not simply zeroed out; instead, their score columns are reused to approximate long-tail attention contributions. This allows each query to use a different block budget while avoiding the rapid degradation in visual quality that conventional keep-or-drop methods can cause at high sparsity levels.
Using identical prompts and seeds, the authors compared Sol-Attn with dense-attention pipelines. Wan 2.1 and Hunyuan 1.5 achieved end-to-end speedups of 2.02× and 2.12×, respectively, for 720p text-to-video generation. LTX 2.3 reached 1.90× for two-stage 1080p generation, while Bernini video editing achieved 2.34×. When combined with kernel fusion and caching in Sol-Engine, NVIDIA reports speedups of up to approximately 5×, but that figure cannot be attributed to Sol-Attn alone.
The code is available in the `sol-engine` branch of the Sana repository, allowing engineering teams to inspect the kernels and pipeline integration. However, the results presented so far primarily come from NVIDIA's internal testing and visual examples. Blind evaluations across GPUs, video lengths, and highly dynamic scenes are still lacking, and the B200 kernels remain under optimization. Key areas to watch next include peak memory usage, batch throughput, latency consistency at low thresholds, and integration with mainstream inference frameworks.