Transformers main branch fixes lost multimodal candidates, preserves complete batch generation results

On November 4, the Transformers main branch included fix #50198 for silently dropped outputs during multi-candidate generation in ImageTextToTextPipeline and AnyToAnyPipeline. The change addresses how inference pipelines assemble results. Applications that generate multiple image captions or visual question-answering answers at once should verify how many candidates they actually receive.

The original report reproduced the issue with SmolVLM-256M-Instruct, a Tesla T4, and a Transformers development build: with num_return_sequences=2, a single input was expected to return two candidates but returned one; batching two inputs was expected to yield four candidates but returned only two. The report also found that the issue occurred in all three return modes—tensors, newly generated text, and full text—so switching the text-output option alone would not avoid it. Reproduction case

The root cause involved two stages. The forward pass did not preserve the candidate dimension in the batch, allowing the pipeline iterator to drop results. Then postprocessing iterated over the number of input texts and truncated the extra candidates. Under the relevant conditions, the fix reshapes generated sequences into a structure containing both input and candidate dimensions, then aligns each candidate with its original prompt. It also adds regression tests for both pipelines. Code diff

The engineering risk is that downstream workflows may run normally while receiving only a subset of the answers. If a system relies on multi-candidate reranking, quality comparisons, or evaluation, missing candidates could change the outcome. This impact is inferred from the defect mechanism; there is no public evidence quantifying how widespread it is. The official documentation explains that a pipeline includes preprocessing, model inference, and postprocessing, and supports batched inputs. Validation should therefore check both model outputs and the pipeline’s final return values.

The small-model tests reported by the PR author passed, but they do not represent every multimodal model. Deployment teams should confirm whether the fix is included in a released version and compare single-input and batched outputs with their own models, checking candidate counts, prompt alignment, and compatibility with single-candidate generation. A main-branch commit does not establish that an installed package already contains the fix.