2,190 Controlled Videos Reveal VLM Counting Limits: More Sampled Frames Still Cannot Guarantee Correct Event Trajectories

A video language model answering “how many times did it happen?” correctly does not mean it actually identified those events. Released on August 6, the [Low-Frequency Trap study](https://arxiv.org/abs/2608.06361) uses 2,190 procedurally generated videos to construct controlled capability surfaces, separately testing spheres colliding with walls, brief flashes, and state transitions involving sustained displays. The researchers independently vary event count and frequency and retain an executable ground-truth temporal trajectory for each video, allowing model-reported timestamps to be aligned event by event instead of comparing only the final integer answer.

Using 80% accuracy as the reliability threshold, Gemini 3.6 Flash can count up to 12 sustained state transitions at 0.5 and 1 Hz, but no positive event count forms a contiguous reliable region for brief flashes. Across all three task categories, its overall exact-count accuracy is just 21.1%, with a temporal-trajectory F1 score of 36.4%; on the flashing task, these figures fall further to 6.4% and 18.6%, respectively. When event count and frequency are both high, only 0.2% of final counts are correct, the number of events reported by the model is about 28% of the ground truth, and timestamp recall is just 18.1%. The primary failure mode in this regime is missing or merging events, rather than hallucinating large numbers of nonexistent events.

Increasing the sampling density raises wall-collision counting accuracy from 19.6% to 29.3%, but the share of reported sequences that exactly match the ground-truth trajectory remains only 3.7%. Different prompts and reasoning formats also fail to consistently expand the reliable operating region. This directly affects API users because [Google’s video understanding documentation](https://ai.google.dev/gemini-api/docs/video-understanding) states that the File API stores video at one frame per second by default, so details in rapid actions may be lost. Simply adding more frames increases token usage, latency, and cost, yet may not fix the model’s ability to accumulate events and preserve temporal information.

For surveillance, sports analytics, and industrial inspection systems, the next step should be to include event timestamps, ordering, and miss rates in acceptance criteria, while establishing operating boundaries for real-world event frequencies. These results measure the complete model and input pipeline, and the controlled scenarios cannot capture occlusion and compression artifacts found in real videos. They therefore should not be interpreted as a universal upper bound for all video models.