生成模型推論系統
GCache Allocates Diffusion Caching Based on End-to-End Error, Cutting LPIPS to 0.0316 on Wan 2.1 at the Same 2.17× Speedup
Rather than comparing only local differences between adjacent denoising steps, GCache estimates the cumulative impact of cache errors on the final image or video. It uses dynamic programming to select cache refresh points, then applies Bayesian optimization to align theoretical weights with actual generation quality.
Adjacent denoising steps in diffusion Transformers contain substantial redundancy in their intermediate features, so methods such as TeaCache and ERTACache reuse residuals to skip some full-model computations. The problem is that local L1 differences do not always correlate with final quality: a residual may appear to change significantly at one step, yet its error may dissipate quickly, while a small error introduced earlier may be amplified along the ODE trajectory. Deciding when to refresh based only on current similarity can therefore allocate the compute budget to the wrong steps.
GCache first derives global error upper bounds for single and repeated cache reuse, multiplying each local residual error by a propagation weight associated with its temporal position. Because the worst-case upper bounds produced by Lipschitz assumptions are overly conservative, the researchers further parameterize the propagation exponent using a Bernstein polynomial. In the inner loop, given a fixed budget of $K$ full refreshes, refresh-point selection is formulated as a shortest-path problem and solved via dynamic programming in $O(KN^2)$. In the outer loop, Gaussian-process Bayesian optimization searches for the coefficients, allowing the selected strategy to directly reduce measured losses such as LPIPS.
The team tested Open-Sora 1.2, CogVideoX-2B, Wan 2.1-1.3B, and FLUX.1-dev on a single NVIDIA A800 80GB GPU. Video evaluations used 946 prompts from VBench, while image evaluations used 30,000 COCO prompts. Generating an 81-frame, 480p video with Wan 2.1 over 50 steps originally took 199 seconds; GCache-slow reduced this to 91.6 seconds, a 2.17× speedup. At the same speed, its LPIPS was 0.0316, compared with 0.1095 for ERTACache. The fast strategy achieved a 3.01× speedup at 66.1 seconds while keeping LPIPS at 0.0828. For FLUX, at the same 2.87× speedup, GCache also reduced LPIPS from ERTACache’s 0.2658 to 0.1825.
The method requires neither retraining the generative model nor an additional error-correction network, making it suitable for offline deployment-policy searches for a fixed model and scheduler. However, the refresh budget is currently fixed, and the local error matrix is precomputed from complete trajectories. Extreme motion or outlier inputs may cause actual trajectories to deviate from the proxy values, while performance measurements are available only for the A800. The paper does not yet link to public code, so engineers must still wait for cross-GPU reproductions, measurements of strategy-search costs, and results across different solvers, quantization schemes, and dynamic inputs.