Transformers Community Reports Missing Granite Speech Output Scaling, Potentially Affecting Sampling and Fine-Tuning

The Hugging Face Transformers community received a report of a Granite Speech numerical anomaly on October 23: the speech model’s text-output logits were eight times larger than those of the Granite language model using the same text weights. The reporter found the discrepancy while comparing the Transformers and MLX implementations, using Transformers 5.20.2, and noted that the development branch showed the same behavior. Technical report

The issue appears to stem from output handling after a multimodal model refactor. According to the reporter’s investigation, before 5.9, the Granite Speech wrapper included the full GraniteForCausalLM, which performed the scaling. In 5.10, it switched to the base GraniteModel and attached lm_head directly, omitting the step of dividing by text_config.logits_scaling. IBM’s published Granite Speech 4.1 2B Plus configuration sets this parameter to 8. Model configuration

This kind of error may slip past tests that compare only generated text. Positive proportional scaling does not change the highest-scoring token at a given step, so greedy decoding may still produce the same text; however, after softmax converts the logits to probabilities, the distribution becomes more concentrated. The report says that at a sampling temperature of 1, the effect is equivalent to using one-eighth the temperature with correct scaling. It may also change beam-search scores, transition scores, and fine-tuning loss. Some evaluations listed on IBM’s model card use four-beam search and a repetition penalty, so those paths warrant separate validation. Report analysis, IBM model card

On October 24, another participant confirmed the eightfold difference in Granite Speech using a small randomly initialized model on CPU, and found similar missing scaling in certain LlavaNext and AyaVision text-backbone configurations. These tests support the presence of a numerical-path issue, but do not yet quantify its effect on speech recognition error rates with official weights. As of this check, the issue remains open, with no fix PR or release listed. Engineering teams can first pin package and model versions, then compare logits, loss, and actual decoding results. Follow-up checks should establish whether a patch covers Speech Plus and whether cross-model output-consistency tests are added. Reproduction and follow-up discussion