模型可解釋性與評估
SAEs Independently Select Their Highest-Activation Tokens, Inflating Cross-Dictionary Causal Variance From Near Zero to 11.9%
A new study finds that sparse autoencoder ablation evaluations typically allow each dictionary to select its own highest-activation token, meaning ostensibly identical latents are tested at different positions. After fixing a shared position, the 7.6% and 11.9% of variance previously attributed to dictionary differences in controlled experiments nearly disappeared.
Sparse autoencoders (SAEs) are often used to decompose a language model’s residual stream into interpretable, nameable latents. To verify whether a latent genuinely affects the model, researchers typically zero out its contribution at the token where it activates most strongly, then measure the KL divergence of the next-token distribution. The problem is that the dictionary being evaluated determines the “highest-activation position.” With a different SAE, even if the decoder direction is nearly identical, the highest-activation token may change, causing cross-dictionary comparisons to conflate dictionary differences with positional differences.
A [new study](https://arxiv.org/abs/2608.13337) first matched Gemma Scope latents released by Google for the same model whose decoder directions had high cosine similarity, confirming that production-grade dictionaries also frequently select different tokens. The authors then trained six SAEs from the same initialization, varying only the fitting configuration to keep latent identity fixed. Across 15 dictionary pairs, 384 sequences, and 3,600 comparisons on Gemma-2-2B, the two dictionaries selected the same token only 13.9% of the time. Although this exceeded the random baseline of 3.5%, the median distance between differing positions was still 118 tokens.
This selection directly changes conclusions about reliability. When each dictionary was measured at its own highest-activation position, latent × dictionary interactions accounted for 7.6% and 11.9% of the variance. When measurements were instead taken at a shared token, both figures fell to nearly zero. Expanding the evaluation corpus sixteenfold did not help, because additional text simply provided more extreme-value positions over which dictionaries could diverge. The authors also found that choices about excluding special tokens and normalizing by intervention magnitude could reverse the direction of comparisons on the same data.
The fix requires only selecting a shared position in the evaluation code, but reporting standards must also specify the position-selection procedure, special-token handling, and normalization method. The [code, preregistration documents, and results](https://github.com/vcnoel/sae-artifact) are publicly available. One important caveat is that the main controlled experiments covered only Gemma-2-2B, Gemma-3-1B, TopK SAEs trained from the same initialization, and a single researcher’s implementation. The conclusions concern effect-magnitude measurements from zero-ablation of individual latents and cannot be directly generalized to interchange interventions, multi-latent concept tests, or all SAE evaluations.