GitHub Repo
Transformers Reports Gemma 3 Attention Compatibility Bug; Flash Attention May Silently Change Image Processing
On November 13, the Transformers community reported that a specific Flash Attention backend drops bidirectional attention for image tokens in Gemma 3. The issue includes a minimal reproduction and output differences, but the impact on full-model quality and the availability of a fix remain unconfirmed.

On November 13, the Transformers community opened a Gemma 3 compatibility issue: when using the kernels-community/flash-attn2 backend, the bidirectional attention originally used for image tokens may be silently changed to causal attention. The program still produces output, but users receive no warning. As of this review, the issue remains open and is not linked to a proposed fix. Original issue
The difference affects which positions the model can access for information. According to the issue analysis, the SDPA path builds an additional mask from token_type_ids, allowing image tokens to attend to one another bidirectionally. The Flash Attention path retains only the padding mask, which makes the image sequence subject to causal constraints. In other words, switching inference backends may also change the model’s computational semantics. Technical analysis
The reporter ran a small test model with Transformers 5.22.1, an H100, and BF16, comparing logits for the same image input across the two backends. The maximum absolute difference was 0.1846; after removing token_type_ids from the SDPA input, the difference fell to 0.0098. This supports the diagnosis that the mask is being dropped, but the values come from a test model and cannot be directly translated into an accuracy drop for the production model. Reproduction code and data
Gemma 3’s multimodal architecture combines a vision encoder with alternating local and global attention layers. The official Transformers documentation’s image inference example explicitly uses attn_implementation="sdpa" and provides an attention mask visualization tool. For deployments involving image question answering and document understanding, these examples can serve as a baseline for comparing backend behavior. Gemma 3 documentation
The immediate engineering takeaway is that evaluations of accelerated backends should check whether multimodal masks are fully preserved, in addition to measuring speed and memory use. For now, teams can establish a baseline using the official SDPA example and then compare results on their actual image tasks. Next, watch for whether maintainers add an explicit error or warning and regression tests for the fix. Full-model quality evaluations and information about an official fix release are still lacking.