Back Home

GitHub Repo

Transformers Proposes Multimodal Parameter Isolation Fix to Prevent Reused Settings from Affecting Later Requests

On November 15, the Transformers community reported that two multimodal pipelines modify caller-supplied generation settings, potentially causing later requests to fail or reuse stale parameters. A fix has been proposed, and another contributor reported validation results, but it has not yet been merged.

Marcus Qwertyus · Public domain · Image source
zh-Hant

On November 15, a report in the Transformers community said that ImageTextToTextPipeline and AnyToAnyPipeline directly modify the caller’s generate_kwargs dictionary while preparing parameters. If an application reuses the same settings across requests, parameters added during an earlier call may affect the next inference. The report used version 5.24.0.dev0 and included reproduction steps with a public model. Issue report

In the specific case, the caller first passes a dictionary containing only do_sample=False, then sets max_new_tokens=2 in the call. The pipeline writes the length limit back to the original dictionary. Reusing the dictionary for a second call with a different length value then raises an exception because the parameter is duplicated. The reporter reproduced this with SmolVLM-256M on an NVIDIA T4. Residual stop strings and a mismatch in generation modality were verified at the parameter preparation layer; these checks are not equivalent to full model output tests. Reproduction details

This type of issue can affect the reproducibility of service wrappers and evaluations. The official documentation describes pipelines as interfaces that combine preprocessing, model inference, and postprocessing, and lists behavior that returns text, audio, or images depending on the generation modality. From this, one can infer that if the generation stage retains an old modality while postprocessing uses new settings, the two stages may become inconsistent. Engineers might also mistake differences caused by request order for model instability. Official interface documentation

The proposed fix, submitted the same day, copies the settings dictionary before adding per-request parameters, while retaining the existing duplicate-parameter check. The submitter said six regression tests passed. On November 16, another contributor reported that after replacing the two pipeline files in version 5.23.0, the parameter-layer reproduction case behaved as expected again. This provides additional evidence, but it is still community testing; as of the time of review, there was no confirmation of a merge or a fix in an official release. Fix and validation

Based on the reproduction results so far, deployments can use a fresh settings dictionary for each call and check whether stop strings or modality settings persist across consecutive requests. Follow-up should track maintainer review, release inclusion, and whether full model tests cover the different output modalities. Current evidence is insufficient to conclude that all multimodal models are affected.

Sources

  1. Multimodal pipelines mutate generate_kwargs, leaking generation settings across requests
  2. fix(pipelines): avoid mutating multimodal generation kwargs
  3. Transformers Pipelines 官方文件