Transformers 5.21.1 Fixes RoPE Configuration Regression, Restoring Loading for Some vLLM Models

Hugging Face released Transformers 5.21.1 on October 29, fixing a model-loading regression introduced in 5.21.0. In the release notes, maintainers explicitly said the previous version broke vLLM’s remote code loading. The new version is also available as a PyPI package, so users do not need to wait for a development build. Release notes, PyPI package

The root cause was that RoPE parameter normalization ran as soon as the rope_scaling attribute was assigned. Some custom configuration classes set this field before completing parent-class initialization, when other parameters might not yet be ready. If normalization added keys to the dictionary too early, validators that strictly check fields could reject the configuration. It could also write a null value before rope_theta had been assigned, or read a maximum position length that did not yet exist. The fix defers this processing until initialization is complete while preserving normalization when parameters are reset afterward. Fix discussion

This kind of error can cross library boundaries. According to the vLLM documentation, service startup begins by reading config.json; some models then use Transformers’ AutoConfig.from_pretrained to create a configuration object. Using a remote custom class also involves configuration code in the model repository. As a result, a change to initialization order at the configuration layer can prevent a service from starting even when the inference engine and model weights are unchanged. vLLM integration documentation

The scope of validation should be clear: the fix author reported 462 related tests passing and 389 skipped, and confirmed that a specified EAGLE3 configuration again produced the correct rope_theta and that a Step5 configuration could be created again. These results support the compatibility fix, but do not mean that every model has undergone end-to-end inference validation, and they provide no evidence of improved throughput. Test record

For teams that upgraded to 5.21.0 and then encountered configuration validation or RoPE initialization errors, 5.21.1 is a patch release worth evaluating. Within existing dependency constraints, engineering teams should rerun cold starts and short inference runs with their actual models, and retain model revision identifiers for comparison. Downstream integration tests and custom configuration classes’ dependencies on initialization order are worth monitoring.