Show-Harness Controls Different Robots with Discrete Semantic Actions, Achieving an 89% Success Rate Across Ten Tasks

Show Lab has released [Show-Harness](https://arxiv.org/abs/2609.10522), which reduces the interface between vision-language models and robots to a readable set of discrete actions, such as `MV_LEFT`, `MV_UP`, `GRASP`, `RELEASE`, and `DONE`. At each step, the model selects a symbol based on multi-view images, proprioceptive state, and a short action history. A robot-specific interpreter then deterministically maps that symbol to a Cartesian displacement, rotation, or gripper command. When switching between Franka, AgileX, or a simulator, the vocabulary presented to the model remains unchanged, while actual step sizes and safe-height parameters are calibrated in the control layer.

This design sits between high-level program agents and end-to-end VLA models: the VLM still determines physical actions step by step but does not need to directly regress high-dimensional continuous control values. The system supports two paths. The first directly invokes frontier VLMs for zero-shot control. The second uses the GUMI graphical interface to collect keyboard or agent demonstrations, then fine-tunes a small open VLM with LoRA. In the paper, Qwen 3.5 2B was trained on 164 real-world episodes comprising roughly 7,800 decision steps, with only about 3% of its parameters updated. Another 230 simulated episodes were used to test sim-to-real transfer.

Across ten real-world pick-and-place tasks, Gemini 3.1 Pro achieved an 89% success rate in the zero-shot setting, while the fine-tuned Qwen 3.5 2B reached 86%; the strongest comparison baseline achieved 57%. In cross-embodiment evaluations involving Franka and AgileX, the corresponding results were 93%, 87%, and 52%. The [project page](https://showlab.github.io/Show-Harness/) also reports that removing multi-view prompting, subtask planning, or proprioception reduced performance by 38, 36, and 28 percentage points, respectively. This suggests that the results come from the complete harness rather than from discrete action tokens alone.

The [open-source code](https://github.com/showlab/Show-Harness) includes real-world and simulation configurations, the LoRA serving pipeline, and safety bounds. However, the authors explicitly require deployers to remeasure their own robot arm’s axis directions, initial pose, and height limits. Each task was typically evaluated in only ten trials, and the scope was largely limited to tabletop pick-and-place involving five object types and two container types. Longer-horizon tasks, contact-rich manipulation, sensor failures, and cross-lab reproduction remain the next real tests of reliability.