Transformers Proposes a torchao Lazy-Import Fix to Prevent Premature CUDA Initialization from Breaking Child Processes

On November 9, the Transformers community reported an import side effect in a dependency: when torchao is installed, importing transformers.modeling_utils can initialize CUDA prematurely, causing later child processes created with fork to fail when they use CUDA. The reporter said the issue appeared in vLLM’s continuous integration tests, where torchao in a shared image affected jobs that did not use its quantization features. Issue report

The reported environment used Transformers 5.22.0, torchao 0.18.0, PyTorch 2.15, and a Linux GPU. The triggering chain starts in the quantizer auto-loading module: it imports a torchao quantizer, which immediately loads experimental safetensors support and indirectly triggers a GPU check. The report includes a minimal reproduction that compares CUDA initialization state before and after the import. For now, the impact should be considered specific to the reported version combination. Reproduction and explanation

The fix proposed on November 10 moves the import of flatten_tensor_state_dict into get_state_dict_and_metadata(), where it is used for serialization. It also adds a regression test in a separate process to check that importing the model utilities does not load torchao or initialize CUDA. The author reported that 30 related tests passed and that outputs were identical after saving and reloading INT8 safetensors. Validation used PyTorch 2.14.1 and did not cover 2.15; full type checking also produced diagnostics in files that were not modified. Proposed fix

The technical issue is when the GPU execution environment is initialized. PyTorch’s official documentation calls the error caused by initializing an accelerator in a parent process before forking “poison fork” and requires CUDA subprocesses to use spawn or forkserver. Lazy imports can eliminate this unintended initialization path, but deployments still need to check how worker processes are started. For teams using shared inference images, the issue also shows that optional packages can change process behavior. Next, track the proposal’s review, the version it is merged into, and validation in the original environment. PyTorch multiprocessing documentation