Transformers 5.20.2 Fixes Lost Input Types in Generation and Includes BF16 Expert Gradient Patch

Hugging Face released Transformers 5.20.2 on October 19, incorporating a fix for input types in the generation pipeline and an expert-gradient patch for a specific distributed-training setup. PyPI published the source distribution and wheel packages the same day, allowing engineering teams to pin the version for validation. Release notes, PyPI package record

The generation issue arose in the new move_to_device path: moving dictionary-like inputs could discard their original container type, affecting even ModelOutput, which inherits from OrderedDict. Later code that relied on attributes provided by the original type could then fail. Maintainers noted that this could affect prepare_inputs_for_generation within generate, particularly in multimodal models. The fix has been merged, with tests added. Input type fix

On the training side, the patch addresses the interaction between BF16, token dispatch, and a specific expert-parallel topology. The patch description gives the condition as tp_size < ep_size < fsdp_size * tp_size: the divisor for expert gradients differs from the size of their FSDP group, leading FSDP2 to use NCCL PREMUL_SUM, which may return zero in the PyTorch versions involved in the report. The release changes the calculation so expert modules sum first, then divide locally. Expert gradient fix

When maintainers reproduced the issue with four B300 GPUs and PyTorch 2.13.0, the BF16 expert gradient norm recovered from zero to 0.0889, close to the FP32 value of 0.0887. They also added a four-GPU regression test. This result supports the fix restoring gradients in that configuration, but does not establish that all topologies and models have been validated. Reproduction data and tests

Engineering teams upgrading should check both input containers in multimodal generation and gradient flow and weight updates for each expert. The first concerns the interface contract after data is moved; the second concerns whether training actually updates experts. Compare results before and after the upgrade using existing workloads. This release provides an official fix, while compatibility with each deployment environment still needs to be confirmed.