機器人／VLA 訓練
EXIMO Uses VLM-Orchestrated Exploration and Distills It into a 3B VLA, Without Additional Teleoperation Demonstrations
Google DeepMind has a vision-language model first break down long-horizon robotic tasks into subgoals executable by a VLA, then fine-tunes the original model on successful trajectories and adds residual off-policy reinforcement learning. The method improves exploration efficiency in simulated tasks, but has yet to demonstrate that it can replace human demonstrations on physical robots.
Google DeepMind’s [EXIMO](https://arxiv.org/abs/2608.19891) divides adaptation to new robotic tasks into three stages—Explore, Imitate, and Optimize—with the goal of fine-tuning a large vision-language-action model (VLA) without adding teleoperation data. The base model used in the experiments is the 3-billion-parameter Gemini Robotics On-Device (GROD). It has basic skills such as picking and placing, but fails on instructions that require common sense or multi-step composition, such as “put the fruit that monkeys like into the bowl.”
During the exploration stage, a stronger VLM is introduced as a closed-loop orchestrator. It periodically inspects scene images and the task history, rewrites the high-level goal into short-horizon natural-language instructions that GROD can directly understand, and replans as execution progresses. A ground-truth success detector in the environment retains only successful rollouts, creating new supervised training data.
The imitation stage does not require continued VLM calls at deployment time. Instead, GROD learns the successful trajectories while conditioned on the original high-level goal, distilling the orchestrator’s decomposition capability back into a single VLA. Finally, EXIMO freezes the large VLA’s primary actions and uses MPO to train a smaller residual policy that outputs corrections to those actions. This avoids the high cost of applying reinforcement learning directly to a diffusion-based action head and gives exploration a better starting point by building on a policy that already has a nonzero success rate.
The paper evaluates exploration and distillation in the ALOHA simulator across 22 manipulation tasks, using 1,000 episodes per method. For the online RL portion, it aggregates results across 20 tasks and five random seeds. The authors report that the complete three-stage pipeline outperforms both GROD without orchestration and a baseline that simply applies RL to the base GROD for longer, measured by success rate, time to completion, and environment steps. [Official GROD materials](https://deepmind.google/blog/gemini-robotics-on-device-brings-ai-to-local-robotic-devices/) show that the original model was already designed for on-device inference and rapid adaptation; EXIMO’s novelty lies in using a VLM to autonomously collect and distill data for new tasks. However, all experiments were conducted in simulation, rely on noise-free success detection and reset mechanisms, and no code has yet been released. The key next questions are sample efficiency on physical robots, the safety implications of failed trajectories, and how much error would accumulate if VLM-based evaluation replaced the ground-truth detector.