AI 代理
Qwen-UI-Agent Unifies Mobile, Desktop, and Browser Operations, With Online RL Trajectories Exceeding 100 Turns
Alibaba’s Qwen team has released a technical report on a GUI agent spanning mobile, desktop, web, and DeepSearch environments. Its unified action space combines on-screen interactions, CLI commands, and batched actions. The model leads existing systems on several self-reported benchmarks, but its weights, training data, and complete execution framework have not yet been released.
The Qwen team has introduced Qwen-UI-Agent, an effort to consolidate today’s largely separate approaches to mobile control, desktop operation, browser tasks, and deep research into a single foundation GUI agent. Rather than relying solely on mouse clicks or touch coordinates, the system interleaves GUI actions and CLI commands within a unified action space and can output a batch of actions in a single model inference. This design can reduce the latency of invoking the model at every step while allowing the agent to handle files or inspect system state directly through the command line when appropriate.
On the training side, the system combines sandboxes with large-scale environments running on real mobile devices. Using an AutoResearch-like data loop, the team has the agent generate tasks, construct environments, analyze failures, and then plan the next round of data collection. Online reinforcement learning handles trajectories exceeding 100 turns, with more than 10,000 parallel environments accelerating rollouts. A lightweight harness also preserves state across devices, enabling the agent to proactively launch services instead of merely responding to individual commands.
The report lists scores of 82.1% on MobileWorld, 92.2% on MobileWorld-Real, 97.5% on AndroidDaily, 79.5% on OSWorld-Verified, 73.6% on WebArena, and 81.5% on ScreenSpot-Pro. On OSWorld-v2, it scores 40.0% under partial-progress evaluation, underscoring that results from different benchmarks and scoring rules cannot be compared directly. For engineering teams, the significance of this work is that GUI and CLI tasks may no longer require two separate agents; instead, they could share a policy, memory, and long-horizon training pipeline.
The primary limitation is that the technical report does not yet include reproducible model weights, training datasets, inference code, or cost data, and the results are largely self-reported by the development team. The next points to watch are whether the team releases practical deployment interfaces, permission-isolation and interruption-recovery mechanisms, and whether third parties can reproduce its long-horizon task performance on the original OSWorld benchmark and real devices.