最新模型
Microsoft Releases Mage-VL-Cowork, Letting Small Desktop Agents Switch Between GUI and Terminal Operations
On October 30, Microsoft unveiled Mage-VL-Cowork, which uses a single vision-language model to choose between mouse-and-keyboard actions and terminal code execution. Microsoft reports a 66.0% success rate on OSWorld-Verified and provides reproduction tools, though results remain subject to the test environment and execution framework.
On October 30, Microsoft unveiled Mage-VL-Cowork, a desktop operation model that combines screenshot understanding, graphical user interface (GUI) interaction, and terminal execution in a single policy. The model uses Qwen3-4B as its language backbone, paired with the Mage-ViT vision encoder, and its weights are licensed under Apache 2.0. The release date is also noted in the model release tracker, while the architecture and license are confirmed by the official model card.
The key change is that the model chooses how to act at each step. It can call computer_use to work with screen layouts and mouse-and-keyboard input, or use bash to directly modify spreadsheets, documents, and presentations, then inspect the resulting files. This could reduce cell-by-cell clicking when handling structured data, with the model itself choosing the route. The official model card describes this dual-channel tool contract.
Training used five stages: first establishing visual grounding and tool-calling capabilities, then separately learning GUI, code, and terminal operations, and finally integrating them through mixed supervised fine-tuning and online reinforcement learning. The team also adjusted the teacher model’s tools, prompts, and execution environment before collecting demonstrations. Mage-CUA-Forge synthesized about 294,000 tasks in total. One aspect of this method worth studying is that it also brings the teacher’s operating framework into data quality control. The official project page describes the training pipeline.
Microsoft reports an average success rate of 66.0% across two runs on 361 OSWorld-Verified tasks, excluding Google Drive, with a limit of 50 steps per task. Compared with the mixed supervised fine-tuning version, online reinforcement learning increased success by 10.16 percentage points while reducing model calls by 13.0% and output tokens by 25.4%. These are the team’s measurements and cannot be directly generalized to other desktop environments or typical workloads. The evaluation details list the comparison conditions.
There is still integration overhead for engineering teams. The official setup uses a custom llava_onevision2 architecture, requires loading remote code, and provides an SGLang GPU container with pinned dependency versions, along with reproduction scripts. The next things to watch are whether third parties can reproduce the results, and how stable the model remains when application versions, terminal packages, or tool permissions change. Microsoft also recommends running it in an isolated container or virtual machine. The deployment documentation provides the setup details.