Transformers 5.20.1 Fixes Loading Regression on Older PyTorch, Adds Beam Model Support

Hugging Face released Transformers 5.20.1 on October 16, following 5.20.0 the previous day. The patch addresses a compatibility regression acknowledged by the maintainers: an earlier change broke model loading on older PyTorch versions. It also adds Reflection AI’s Beam model, combining integration of a new architecture with a fix for existing environments. Official release notes

The loading fixes address two separate issues. The release notes say the from_pretrained issue on PyTorch 2.7 was addressed by restricting distributed checkpoint functionality to PyTorch 2.8 and later. A separate change makes DtensorShardOperation work with PyTorch 2.6 and 2.7. Teams pinned to older runtime environments should verify standard weight loading, tensor sharding, and distributed checkpoint paths separately; a successful package installation alone does not establish compatibility. Fix details

For inference, the release lists a fix for DeepGEMM Mega MoE with expert parallelism, along with a fix for image assignment during HunYuan-VL batched generation. The former affects multi-GPU expert computation, while the latter concerns how images in a batch are matched to inputs. The release notes do not provide a comprehensive throughput comparison after the fixes, so they do not establish that the previously reported expert-parallel performance regression has been fully resolved. Release notes

PyPI provides the 5.20.1 source distribution and wheel, both dated October 16. The package page continues to list support for Python 3.10 and later, and PyTorch 2.6 and later. This gives deployments using older PyTorch versions a patch release they can install, although individual features may require newer versions. PyPI package record

Engineering teams should next verify cold starts, sharded loading, and first-generation behavior with their existing model checkpoints, then measure latency and throughput in expert-parallel deployments. Teams using vision models should also test batches containing multiple distinct images to confirm that outputs correspond to the correct inputs. These checks can establish whether the fixes cover the paths used in production.