GitHub Repo
Transformers Proposes Multimodal Pipeline Fix for Rejected Image-Only and Audio-Only Inputs
A November 17 Transformers community report says AnyToAnyPipeline requires text, blocking image-only and audio-only inference supported by underlying processors. A proposed fix submitted the same day includes cross-model tests, but has not been merged; its impact on released versions remains unclear.
Transformers’ unified multimodal interface has an input validation gap. On November 17, a contributor reported that when using the AnyToAnyPipeline in the 5.24.0.dev0 development build, the pipeline may reject requests without text even when the underlying processor accepts image-only or audio-only input. A fix was submitted the same day. Issue report
The reproduction uses a miniature Gemma3n checkpoint to create a pipeline. Passing only images triggers a type error because the text argument is missing; passing text=None with audio is blocked by validation requiring text or images; and an audio dictionary input also fails because no text key is found. The author traces the issue to four areas: the call signature, media validation, preprocessing, and postprocessing for empty prompts. Reproduction code and analysis
The official documentation describes this pipeline as combining a model with a composite processor and providing images, videos, audio, and text parameters, but the documented call signature still makes text mandatory. From an interface design perspective, this kind of restriction makes it difficult for applications to distinguish model capabilities from wrapper limitations. When diagnosing media input failures, engineers should check the processor and pipeline paths separately. Official API documentation
The proposed fix makes text optional, expands media checks, and handles empty prompts. The author says they completed seven media-generation tests using miniature checkpoints for Gemma3, Gemma3n, Gemma4, GOT-OCR2, Pix2Struct, and BLIP; six newly added regression tests also went from failing to passing. These results verify that the interface can complete generation; they do not establish the output quality of full models. Proposed fix
As of this review, the proposal remains open and has no review record. Deployment teams should first pin a version and reproduce their own input formats, then track the merge and release status. In particular, they should confirm whether text, chat, and batch requests remain compatible. The evidence comes from a development build and tests run by the submitter, so it is not enough to claim that all stable versions or multimodal models are affected. Proposal status