ComponentBench Breaks Down 2,910 Web Tasks, With the Same Agent Varying by More Than 34 Percentage Points Across Interface Modes

The newly released ComponentBench reduces the evaluation unit for computer-use agents to a single UI component, filling the gap between atomic tests such as “clicking coordinates” and complete workflows spanning dozens of steps. The full version includes 2,910 tasks, 97 standard components, and 14 interaction families, covering button groups, date and time inputs, rich-text editors, data tables, sliders, and cross-region drag-and-drop. The components come from real-world React libraries such as Ant Design, MUI, and Mantine rather than static screenshots.

Each task has programmatic success criteria and a cleaned human interaction trajectory. The evaluator marks a task as passed only when the DOM reaches the specified state; it does not trust an agent’s self-reported “completion” or rely on another LLM as a judge. The research team has also released the task YAML files, JSON Schema, execution framework, step-by-step logs, screenshots, and failure cases, allowing developers to determine whether a problem stems from visual recognition, target localization, action selection, state verification, or premature termination.

Results from seven models across four observation/action interfaces show that the agent harness and interface representation can be enough to change model rankings. GPT-5 mini achieved an 83.1% success rate in accessibility tree mode, falling to 48.9% when switched to pixel-only coordinate control. Qwen3-VL-235B similarly dropped from 77.0% with the AX tree to 50.5% in Pixel mode. The overall leader was Gemini 3 Flash paired with Browser-Use, with a 95.2% pass rate, but even the fastest configuration took 3.7 times as long as the human reference time. Drag-and-drop, high-precision continuous input, and advanced editors were the most difficult component families.

This suggests that a single aggregate score is insufficient to predict product performance when procuring or fine-tuning computer-use models; DOM, accessibility tree, annotated screenshot, and vision-only modes are effectively different system designs. One limitation is that the tasks still focus primarily on short React web interactions and cannot cover authentication, network latency, cross-page state, or error accumulation in long workflows. Going forward, it will be worth watching whether the smaller but more difficult 912-task Core version can remain stable across releases, and whether the published trajectories can drive training improvements targeting drag-and-drop and state verification.