GitHub Repo
Transformers Community Reports T5 Attention-Switching Failure: Configuration Shows SDPA, but Eager Still Runs
A development-build reproduction reported on October 6 suggests that switching T5’s attention backend after loading may update only the top-level setting, while the encoder and decoder continue using eager. The report includes direct call counts, but maintainers have not confirmed the issue and no fix has been merged.
On October 6, the Hugging Face Transformers community reported a possible mismatch between configuration and actual computation when switching T5’s attention backend at runtime: after loading with eager and calling set_attn_implementation("sdpa"), the top-level configuration indicates that the switch succeeded, but the internal encoder and decoder continue using the original backend without warning. The report used development build 5.19.0.dev0, PyTorch 2.11.0, and an ARM CPU on macOS. The impact on stable releases remains unconfirmed. Community report
The reporter ran a single forward pass with google-t5/t5-small, intercepting and counting calls to the SDPA function in the attention registry. The path that switched backends after loading recorded zero calls; the control loaded directly with SDPA recorded eighteen. This provides more direct runtime evidence than reading the configuration, but speed, memory, and generation-quality differences have not been measured. Reproduction code
The author points to a condition in how submodel configurations are propagated: the code decides which objects to update based on whether their configuration classes differ. T5’s two stacks hold deep copies of the configuration, but both use the T5Config class, so they may be skipped. This remains the reporter’s root-cause analysis; other models have not been checked individually. Mechanism analysis
The official documentation lists runtime switching as a supported feature and explains that attention backends improve memory traffic through different computation methods. For engineering teams, this kind of mismatch could cause backend-comparison experiments to be labeled with the wrong conditions, or prevent regression tests intended to exercise SDPA from reaching that path. These are testing risks inferred from the case; they do not establish that GPU acceleration is broadly broken. Attention backend documentation
For now, teams can refer to the direct-loading approach in the report and verify the backend with call tracing. Next, watch for maintainers to confirm whether configuration propagation is at fault, whether a fix covers independent configuration objects of the same class, and which stable releases require test updates. As of this check, the issue remains open, and the page lists no fix branch or PR. Issue status