GitHub Repo
Transformers Community Reports Audio Model Cache Retaining References, Repeated Loads May Accumulate GPU Memory
A November 16 Transformers issue report says method caching for Xcodec and Higgs Audio V2 Tokenizer may prevent model objects from being garbage-collected. The report includes a minimal CPU reproduction, but the full scope of impact and the status of an official fix remain unconfirmed.

The Transformers community reported an audio model memory-reclamation issue on November 16. A user said that after running certain length calculations with Xcodec and Higgs Audio V2 Tokenizer, the objects may remain alive even after deleting model references and running garbage collection. The reported environment used Transformers 5.15.1, and the user said the same code is present on the main branch. This is the user’s diagnosis; it does not establish that all versions are affected. Issue report
The focus is lru_cache on _get_conv1d_layers. The official Python documentation explains that this type of cache retains references to arguments and return values until entries are evicted or the cache is cleared. When applied to an instance method, the arguments also include self. As a result, deleting the caller’s variable does not necessarily remove every reference to the model; the cache may extend the object’s lifetime. Python documentation
The report provides a CPU test that does not require downloading weights. It compares garbage-collection results using a randomly initialized model and a weak reference: the model is released when the length calculation is not called, but remains alive after it is called. The user also said that repeated loads in a live speech service left about 770 MB of FP32 codec weights behind each time; after clearing the method cache, displayed memory usage stayed stable across six reloads. These figures have not been independently verified. Reproduction and observations
This code path is directly relevant to audio services. Hugging Face’s official example uses HiggsAudioV2TokenizerModel.encode() to convert audio into codes, then decode() to reconstruct it; the documentation positions the model for real-time or large-scale audio processing. In deployments that unload models between tasks to free capacity, persistent references could lead capacity planning to underestimate memory requirements during long-running operations. Model documentation
The discussion proposes removing the cache or switching to an instance-level cache. Engineers should watch for whether reclamation tests cover both models, whether a fix is merged, and what repeated load-and-unload measurements show. Based on the reference-retention mechanism described above, a single inference peak is not enough to identify this kind of issue; deployment validation should also track the memory baseline after multiple task cycles. Discussion of possible fixes