Vision Models Can Zoom In on Images Without Necessarily Using the Evidence; Qwen3-VL-8B’s Net Gain Is Just 6.9 Percentage Points

Multimodal models often process details by cropping and enlarging parts of an image, but calling a tool does not mean the returned image actually affects the answer. The Shanghai AI Laboratory and other institutions propose a three-layer causal audit: first, compare accuracy with tools enabled against direct inference; next, replace every returned result in complete trajectories with a random crop from another location; and finally, hold the existing prompts and tool actions fixed while replacing the observation at a single step, measuring the change in the probability gap between the correct option and a distractor. This Visual Evidence Gain metric attempts to isolate the effect of “seeing new evidence,” preventing answer shifts caused by the textual representation of tool actions themselves from being counted as visual reasoning.

The paper evaluates DeepEyes, Pixel Reasoner, Mini-o3, Qwen3-VL-4B/8B, and Thyme across five fine-grained visual benchmarks. The effect of tool mode relative to direct inference varies widely: DeepEyes shows almost no gain; Mini-o3 improves by 21.3 percentage points on VisualProbe, but this does not imply that every crop is effective; and Qwen3-VL-8B posts a net gain of 6.9 points on V*. Within that result, trajectories classified as Calibrated contribute a positive 7.6 points, while the other groups offset part of the improvement.

The authors divide failures into “calling without looking” and “looking without planning.” In the former, replacing the returned image leaves answer preferences almost unchanged. In the latter, the model responds to visual content but continues zooming after it is already sufficiently confident in the answer, or repeatedly crops uninformative regions. Among Mini-o3 failures of this kind, 38% of crops cover less than 4% of the original image. The direct implication for agent evaluation is that tool necessity, evidence sensitivity, stopping strategy, and token cost should all be recorded, rather than comparing only final scores.

The study is limited to open-weight models and crop-and-zoom tools. Its step-level method also requires logits, so it cannot be directly applied to closed APIs. The authors suggest that outcome-based reinforcement learning may produce flawed tool-use habits, but they have not yet established causality through controlled training. In addition, the GitHub repository currently contains only a single commit and a README/license file, which is not enough to fully reproduce the experiments.