GitHub Repo
Transformers Community Reports FSDP2 Loading Memory Bug; Multi-GPU Startup May Create Duplicate Full Models
A community report involving Transformers 5.12.0 says that after enabling the CPU memory-efficient loading option, other training processes may still allocate a full model, exhausting host memory. Official documentation says the option should prevent duplicate loading across processes, but the root cause and fix status await upstream confirmation.