GitHub Repo
Transformers Proposes Unified Quantization Architecture; MXFP4 Migration Could Change Loading Behavior on Older GPUs
On October 20, Transformers maintainers proposed a shared block quantization architecture covering formats such as FP8 and MXFP4, along with mixed-precision loading. The MXFP4 draft also raises the hardware requirements for quantization kernels, potentially changing execution on A100, T4, and CPU; it has not yet been merged.
Hugging Face Transformers maintainer IlyasMoutawwakil submitted a quantization refactor draft on October 20 to let FP8, MXFP4, NVFP4, and MXFP8 share a common loading and computation foundation. The core proposal is still a Draft and does not register any formats itself, so it should not be considered a newly released feature. Shared architecture proposal
The architecture centers on BlockScaledConfig and BlockScaledQuantizer to unify block-scaled formats. FineGrainedConfig’s dtype_plan then assigns different formats based on module name patterns. The proposal also handles the layout of MoE expert weights and scale values: if different kernels require different layouts, switching between them raises an error to prevent weights from being paired in the wrong order. For engineers maintaining mixed-precision models, this offers a way to describe loading rules centrally. Architecture and layout design
A dependent MXFP4 draft plans to support W4A16, W4A8, and W4A4, and to bring GPT-OSS and ModelOpt checkpoints into the shared path; W and A refer to weight and activation precision, respectively. However, the draft explicitly states that all MXFP4 formats will require a GPU compute capability of at least 8.9 because the kernels use Triton’s FP8 type. On A100 and T4, GPT-OSS will instead be dequantized to BF16, and CPU execution will also use dequantization. MXFP4 migration draft
This compatibility change is worth tracking in advance. Current official documentation still lists a minimum compute capability of 7.5 for MXFP4 and says that supported environments automatically download and use quantization kernels. If the draft is merged as written, memory requirements and performance could change for some existing deployments. That is an inference from the change in execution precision; there are no benchmark results for this draft yet to quantify the impact. Current MXFP4 documentation
Engineering teams should track whether the hardware fallback rules change, when proposals for each format are merged, and what migration guidance appears in the official release. Evaluation should check post-loading precision, peak memory, and generation speed; the four-bit format label on a checkpoint alone does not determine its actual execution cost.