Transformers Confirms Processor Parameter Override Issue; Mixing Template Parameters May Disable Input Truncation

Hugging Face Transformers received a bug report on October 20 affecting multimodal preprocessing: when calling ProcessorMixin.apply_chat_template() with both processor_kwargs and extra parameters not referenced by the template, the specified truncation settings may be overridden. The reported environments include the stable release 5.20.2 and main branch version 5.21.0.dev0; project member zucchini-nlp confirmed the issue.Issue report and maintainer discussion

The reproduction loads the processor for Qwen2-VL-2B-Instruct, sets max_length=5 and truncation=True in processor_kwargs, and enables tokenization. The first output has length 5; in the second run, adding only enable_thinking=False increases the length to 20. The report says the model’s template does not use this parameter, so the code classifies it as a processor parameter and replaces the original dictionary, ignoring the length and truncation settings in the process.Reproduction

This interface sits in the model input preparation stage. The official documentation explains that multimodal Processors handle chat templating, text tokenization, and media preprocessing, then pass the results to the model for generation. The API also lists processor_kwargs and defines truncation and max_length as text-processing options. Therefore, engineering checks should cover the actual input length after preprocessing, not just the output limit on the generation side.Multimodal workflow documentation, Processor API

Based on this mechanism, training or inference wrappers that share template parameters across models may unexpectedly increase input length after switching models, affecting batch memory requirements and consistency of data truncation. However, the current example uses only a text message, so it does not establish that all vision models or media parameters are affected.

The maintainer discussion notes that compatibility with legacy extra parameters and precedence when the two sets of parameters conflict will both affect the fix. Engineers can first compare input_ids lengths for the models they actually use, with and without extra template parameters, and track subsequent merge rules and official releases. For now, the confirmed issue should not be taken to mean a fix is already available for deployment.Discussion of possible fixes