Video Models Score Just 0.2% in High-Frequency, Multi-Event Regions; More Sampled Frames Still Fail to Recover Event Evidence

Video language models can score well on aggregate benchmarks yet still fail to reliably record repeated events. A University of Maryland team expanded Low Frequency Trap by creating 2,190 synthetic videos, each 24 seconds long, depicting balls bouncing off walls, brief flashes, and persistent state transitions. Event counts range from 0 to 12, while frequencies range from 0.5 to 4 Hz. The renderer also outputs event timestamps, states before and after each event, and cumulative counts, allowing researchers to verify every event listed by a model rather than checking only its final answer.

At an 80% reliability threshold, Gemini 3.6 Flash correctly handled up to 12 persistent state transitions at 0.5 and 1 Hz. For brief flashes, however, it failed to establish a reliable region even at the lowest positive event count. When both event count and frequency increased, final-count accuracy fell to 0.2%, while recall of ground-truth events was 18.1%. Qwen3‑VL‑235B, evaluated in supplementary tests, likewise concentrated its successful results in low-count, low-frequency regions. However, because the two models used different native interfaces and sampling rates, their charts cannot be used for direct performance comparisons.

More importantly for engineering teams, increasing the ball-bounce video input from the native setting to 4 FPS raised final-answer accuracy from 19.6% to 29.3%, yet the F1 score for aligning the model-reported event sequence with the ground truth was only 3.7%. Selecting event-centered keyframes pushed answer accuracy as high as 68.6%, but the corresponding trajectory F1 remained just 9.7%. This suggests that providing more visual evidence may help a model guess the correct total without demonstrating that it actually identified the correct events.

For video search, surveillance, and process-auditing systems, deployment tests should record event timestamps, recall, and the proportion of cases in which the answer is correct but the supporting evidence is unfaithful. The research currently relies mainly on three simplified animations and Gemini as its sole primary model, and it has not yet isolated the effects of occlusion, camera motion, or irregular event intervals. The new version is also a substantial expansion of earlier workshop research rather than an entirely new line of inquiry.