Transformers Community Reports Ignored FSDP Exclusion Setting, Potentially Affecting Fused Loss Kernel Integration

On October 29, the Transformers community filed issue #50141, reporting that when ignored_modules is passed through TrainingArguments via fsdp_config, the setting is not included in the arguments passed to Accelerate. The report uses Transformers 5.22.0.dev0 and includes a minimal reproduction that does not require a GPU: after specifying lm_head for exclusion, the corresponding field in fsdp_plugin_args still reads None. Issue and reproduction code

This field affects how a model is sharded. FSDP2 distributes model parameters, gradients, and optimizer state across multiple GPUs, then gathers the parameters needed for computation, trading communication costs for memory capacity. If engineers deliberately keep some modules unsharded, whether the setting takes effect can affect downstream code’s assumptions about tensors. Transformers FSDP2 documentation

The reporter’s use case is to keep the frozen output layer lm_head unsharded so a fused linear operation and cross-entropy kernel can receive ordinary tensors instead of encountering DTensors. The author points to _process_fsdp_args as the cause: it forwards only a fixed set of fields, omitting ignored_modules, and does not warn about unknown keys. This is a concrete integration risk, but the public reproduction checks only that the setting is passed through; it does not demonstrate a failure in full distributed training or a performance regression. Report details

Accelerate’s official API documentation confirms that its FSDP plugin supports this parameter. String values are matched against module names using full regular-expression matching, and FSDP2 then converts matched modules into parameters to exclude. Therefore, troubleshooting should confirm both that the field is forwarded and that the actual module names match. Accelerate API documentation

PR #50144, opened the same day, proposes forwarding the parameter and adding field documentation and a unit test. At the time of review, the PR remained open with no review outcome, and its page showed CI waiting for maintainer approval. Engineers can first check the settings actually received by the plugin and verify parameter types after model preparation. They should then track whether the fix is merged, which release includes it, and regression testing across multiple GPUs with the fused kernel. Proposed fix