Transformers Community Reports HunYuan-VL Batch Generation Bug That May Pair Candidate Sequences with the Wrong Images

On October 11, the Transformers community reported a HunYuan-VL multimodal generation bug: when a batch contains prompts with different images and each prompt requests multiple candidate sequences, the generation process may fail to preserve the correct image-to-prompt mapping. The report covers both beam search and sampling, suggesting the issue may lie in an input expansion step shared by the two strategies. Public report

The minimal reproduction uses a small model from the test suite and prepares two prompts, each with an image. The author first obtains the logits for the first step of greedy decoding, then generates with two beams and with sampling, requesting two sequences per prompt, and compares the first-step logits for each candidate. Both paths show a maximum difference of about 0.0967, rather than the near-zero difference the author expected. The test generates only one token, so it measures score differences before candidate sampling, not text changes accumulated over a long answer. Reproduction code and output

The author points to a data-structure contract mismatch as the cause: get_image_features returns a tensor containing all images, while the generation process’s replication logic expects one tensor per image. As a result, text candidates and visual features may become misaligned after batch expansion. The official documentation explains that beam search maintains multiple candidate sequences, while sampling selects tokens from a probability distribution; generation preprocessing also includes batch expansion. From an engineering perspective, this suggests multimodal systems must expand text and images in sync, or changing the decoding configuration may change the conditions the model sees. Generation strategy documentation

This report is relevant to validating batched image captioning, document understanding, and workflows that generate multiple answers for reranking. Engineers can compare single-item and batched runs with two visually distinct images to check whether candidates remain associated with their original inputs; comparing first-step logits could also serve as a regression test. These are validation suggestions based on the reported case; no public data quantifies any accuracy loss in production models.

The current evidence comes from a small test model on 5.19.0.dev0, and another participant said they could reproduce the issue. At the time of review, the issue remained open, the author said they were preparing a fix, and the page did not link to a patch PR. Follow-up should confirm whether the fix covers different numbers of images, both generation strategies, and production weights. The available results are not enough to conclude that all HunYuan-VL deployments are affected. Discussion and fix status