機器人／世界模型
GeniWorld Renders Robot Actions as Visual Conditions, Raising Real-World Success Rate to 69.0% With Synthetic Data
GeniWorld first uses URDF to convert joint actions into pixel-aligned robot motion, then employs a causal DiT to predict how the environment will respond. Across four real-world tasks, the authors report that the overall success rate increased from 40.8% to 69.0%, although neither the code nor model weights have been released.
Most robotic world models feed joint angles or end-effector coordinates directly into a video generator. However, these low-dimensional values lack pixel-level spatial correspondence and can easily conflate robot-body motion, object changes, and background appearance within a single conditioning pathway. GeniWorld, announced on August 6, instead uses the robot’s URDF and forward kinematics to render future actions as dense visual sequences containing only the robot body. After encoding them with a 3D VAE, it concatenates them channel-wise with the video latent and feeds the result into a causal DiT, which uses flow matching to predict the next observation segment.
This design separates embodiment kinematics from scene dynamics while preserving the spatiotemporal priors of a pretrained video model. During inference, the model autoregressively feeds generated frames back into the control loop and retains history using a KV cache. The authors trained it on 2,250 clean-scene trajectories spanning 50 RoboTwin2.0 tasks, then conducted zero-shot testing on 250 randomized-scene trajectories. The visual-action version achieved an FVD of 20.15, compared with 35.85 for Ctrl-World. Reducing flow-matching sampling from 50 steps to five reportedly increased speed by about 10× while degrading FVD by only around 2%.
The real-world evaluation covered moving a bowl, folding a towel, placing a cup, and opening a drawer. For each task, the researchers first collected 25 human demonstrations, then added synthetic trajectories featuring spatial randomization and diverse scenes. The overall success rate rose from 40.8% with real data alone to 69.0%. Five-step sampling reached approximately 8 Hz on a single NVIDIA H20, which is sufficient for interactive teleoperation but still falls short of high-frequency physical control. More importantly, the current evidence covers only four bimanual tabletop tasks, and success assessments partly rely on human judgment and a VLM. The project page currently presents only the method and results; the code, weights, and data remain unavailable. Its generalization and real-world relevance therefore still require independent verification.