GitHub Repo
Transformers Proposes MXFP8 Expert Decompression Fix for MoE Group-Scaling Load Error
On November 5, a Transformers maintainer proposed a fix that restores FP8 expert weights through the compressed-tensors decompressor, addressing an incompatibility with MXFP8 group scaling. The proposal has not been merged, and currently has only a single model validation result reported by its author.
Transformers maintainer ArthurZucker opened PR #50387 on November 5 to fix how FP8 mixture-of-experts model weights are decompressed in the compressed-tensors integration. At the time of review, the proposal was still open, so it should not be treated as a fix available on the main branch or in an official package. The change addresses compatibility when loading quantized weights and is directly relevant to engineering teams deploying MoE models themselves.
The author says the existing expert-weight handling directly computed quant * scale, which causes problems with group scaling. The proposal gives MXFP8 as an example: one scale factor, stored as uint8 in UE8M0 format, is used for every 32 values. The fix routes each expert through the compressor, reusing the decompression method used for dense layers. This issue illustrates that a loader must correctly interpret the scale data format and grouping rules as well as recognize FP8 weights. Author’s explanation
For validation, the author used INCModel2/Qwen3-30B-A3B-Instruct-2507-MXFP8-FP8ATTN-CT-AutoRound and reported that loading failed before the fix. After the fix, the relative error in logits compared with a BF16 reference was 0.046. However, the page does not specify the full test input, the error calculation method, or downstream task results. This value should therefore be treated only as a limited validation reported by the author; it does not establish Chinese generation quality or compatibility with other models. Validation summary
The official documentation says compressed-tensors supports multiple weight precisions and scaling strategies, including tensor, channel, group, block, and token scaling. During loading, weights can also be decompressed to the model’s data type, or kept quantized for execution on supported paths. These differences mean that evaluations of quantized models should record the quantization settings, loading parameters, and package versions; the “FP8” label alone does not determine how a model runs. Official documentation
The next steps are to track integration tests, review, and the merge outcome. Teams using the same format can first create a small loading and output-comparison case, then validate it on their actual Chinese-language tasks. Public evidence is currently insufficient to claim that all MXFP8 MoE models are affected or that this fix improves inference speed.