Transformers proposes mixed-quantization loading fix to prevent random initialization of unquantized weights

On November 7, the Transformers community proposed PR #50466, a fix for the weight-loading path for mixed-precision compressed-tensors checkpoints. The issue was reported on November 6: some modules that were supposed to retain unquantized weights had quantized-weight conversion rules applied to them. As a result, the source tensors were not loaded and the target parameters were instead randomly initialized. As of the time of review, the proposal had not been merged. Issue report

The reported case involves RedHatAI/GLM-5.3-Flash-NVFP4. Its model card says NVFP4 quantization applies only to the linear-operation weights and activations of MoE experts, rather than using four-bit precision uniformly across the entire model. Such checkpoints require the loader to preserve different representations on a per-module basis; the quantization format in a model’s name does not indicate how every tensor is stored. Model card

The issue arose because the quantizer rewrote the source names used by weight converters based on the configuration for the entire checkpoint. The attention Q, K, and V convolution-fusion path, which should have read .weight, instead searched for companion tensors such as weight_packed and weight_scale, and therefore missed data still stored as ordinary weights. The reporter reproduced the issue with version 5.22.1 and a development version. The loading report showed source weights as UNEXPECTED and fusion targets as MISSING. Reproduction and environment

The proposed fix first reads the actual tensor names from the shard index or safetensors header, then determines whether each converter needs to dequantize. Sources containing ordinary weights retain the original conversion path. The author also added error checks for cases such as missing scale tensors, and reports that all eleven new tests passed, covering mixed quantization, INT4 decoding, and save-and-reload behavior. Fix proposal

For deployment teams, this is a reminder that quantization validation should also check weight consistency after loading. Based on the reported mechanism, engineers should investigate the loading warnings and compare the fused parameters with their source tensors. Available evidence does not quantify any accuracy loss in the full model and cannot be generalized to other inference frameworks. Follow-up should track maintainer review, full integration testing, and inclusion in an official package release.