Transformers Community Reports Qwen4-Exp Cache Reset Gap That May Change Sparse Attention Results

On October 19, the Transformers community reported a Qwen4-Exp cache lifecycle issue: after calling reset() on a DynamicCache or StaticCache, passing the same object to a new request may leave the previous request’s position_ids in place. When QSA actually prunes attention key blocks, this position information may change block selection and output logits. The report tested source code version 5.21.0.dev0, so it does not establish that all production deployments are affected. Issue report

This issue concerns how inference services manage caches. The official documentation explains that KV caches avoid recomputing values during token-by-token generation; DynamicCache grows with the sequence, while StaticCache preallocates capacity to support compilation optimizations. Developers can also create caches themselves and pass them to the model through past_key_values, so whether a cache reset fully clears state affects workflows that manage cache objects manually. Official cache documentation

The reporter used CPU, FP32, and a small randomly initialized model. They first supplied three tokens, then reset the cache and supplied twelve more; the cache then contained twelve tokens, but the position count was fifteen. The test deliberately reduced the sparse selection budget so that even a short sequence triggered pruning; in one comparison, the maximum absolute difference in logits was about 0.031. Removing the leftover position attribute made the result match that of a newly created cache. However, the difference cannot be directly translated into a decline in answer quality, and the single token produced by greedy generation did not change. Reproduction and comparison

The proposed patch removes the position attribute after resetting each layer’s state. The author said local checks passed, but CUDA, compiled execution, and pretrained weights have not been tested. In the test, ordinary standalone generate() calls created a new cache and did not reproduce the issue. Engineering teams that reuse caches themselves should watch for how maintainers define responsibility for clearing metadata, and compare reset and newly created caches with their actual models. There is no official fix release yet. Proposed patch and limitations