代理系統與評測
Increasing the Inference Budget for Local Computer Agents Does Not Necessarily Improve Accuracy, With Benefits Reversing After Four Historical Screenshots
An OSWorld experiment shows that giving local GUI agents a small amount of screen history can substantially reduce repeated actions, but further extending the context or step limit mainly changes the types of failure. A two-stage planning architecture also trails a single agent because of formatting and coordination overhead, while generating more plans in parallel recovers only part of the gap at a high token cost.
Researchers used 361 Ubuntu tasks from OSWorld to test Qwen3-VL-8B, Qwen3-VL-30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B, systematically varying four inference-time resources: the number of historical screenshots retained, the number of permitted action steps, whether planning and grounding were split into two stages, and the number of candidate plans generated concurrently. All agents relied solely on screenshots, without using the accessibility tree, and inference was performed with vLLM on a single A100 80GB GPU.
The clearest gains came from adding “a little history.” The average success rate rose from about 18% with no retained screenshots to more than 25% with one. For Qwen3-VL-30B-A3B, retaining four historical screenshots produced a success rate of 28.56%, but increasing that number to eight reduced it to 27.16%. History lets agents know what they have already clicked, reducing loops and failures caused by exhausting the step limit, but it does not improve completion judgment. As the context grows longer, failures increasingly take the form of agents declaring success prematurely.
Increasing the maximum number of action steps from 15 to 50 or 100 produced the same pattern: agents hit the step limit less often, but task success rates remained broadly unchanged while cumulative prompt-token usage and action costs continued to rise. This suggests that erroneous trajectories generally do not correct themselves simply by taking more steps. The more complex “planner–grounder” architecture performed worse overall than a single agent, for reasons including mismatches between plans and screenshots, missing required fields, and unparseable output formats. Generating four candidate plans at once avoided some formatting errors, but the improvement was smaller than the additional compute cost.
The engineering takeaway is that local agents should not directly adopt test-time scaling recipes developed for frontier large models. More practical approaches include retaining a filtered short-term state, adding loop and stagnation detection, and using an independent verifier to confirm that a task has actually been completed. The study is limited to a single benchmark, Ubuntu tasks, and one GPU configuration, and it does not introduce a new training method. Its findings describe resource allocation for current local models rather than a universal law for all GUI agents.