Back Home

GitHub Repo

Transformers Main Branch Adds Video Data Checks to Prevent Silent Batch Truncation

A November 4 commit to the Transformers main branch adds checks for video and metadata counts, addressing cases where videos in a batch could be silently dropped. The update also makes processors that rely on frame timestamps explicitly reject disabled metadata; whether the fix is included in a released package remains unconfirmed.

Marcus Qwertyus · Public domain · Image source
zh-Hant

The Transformers main-branch commit log listed video preprocessing fix #50060 on November 4, with commit ID f275431. The change addresses two input validation issues: mismatched counts of videos and metadata, and processors that require frame timestamps receiving return_metadata=False. The proposal was first made public on October 28; the news is that the fix now appears in the main-branch commit log. Main branch log

The first issue stems from pairing videos and video_metadata with zip. If the metadata list is shorter than the video list, pairing stops when the shorter list ends, potentially skipping the remaining videos. The fix adds a length check and raises a ValueError when the counts differ, with a message that each video must have one metadata entry. By this mechanism, a custom pipeline that receives three videos but only two metadata entries should catch the missing input earlier. Fix proposal and code snippet

Another change concerns processors such as Qwen3-VL that use frame timestamps. The proposal changes the behavior when metadata is explicitly disabled to return a readable error message, avoiding a later KeyError caused by missing timestamps. This does not mean that all video models must return metadata; the requirement depends on whether the processor needs the relevant fields. The official Qwen3-VL documentation says its video understanding uses text timestamps for alignment, making temporal information part of the input semantics. Qwen3-VL documentation

This check is especially useful for teams that decode videos themselves and then pass arrays to a model. The official documentation recommends providing metadata such as the original frame rate and duration when decoded videos still need frame sampling according to the model's strategy. Videos passed directly as paths or URLs can be decoded by the processor. Engineers can first check batch counts and data ordering, and confirm that error handling in upstream code can catch the new validation failure. Video processor documentation

What is confirmed so far is the main-branch log; that alone does not establish that a routine package upgrade includes the fix. Follow the release tags and test multi-video inputs and timestamp settings with the processor actually in use. Public information also does not provide performance or accuracy comparisons for this fix.

Sources

  1. Transformers 主分支提交紀錄:影片中繼資料驗證修補
  2. PR #50060:Require one metadata entry per video and refuse return_metadata=False
  3. Transformers Video processor 官方文件
  4. Transformers Qwen3-VL 官方文件