世界模型與生成影片
ReWorld Uses a Pose-Indexed Landmark Bank to Fix the KV Budget, Allowing Interactive World Models to Return to Earlier Scenes After Long Journeys
ReWorld assigns short-term action control and long-term scene memory to different attention ranges, then preserves history in a fixed-capacity landmark bank. The authors demonstrate real-time streaming at 704×1280 and 64-second return trajectories, but only the paper and demo page are currently available; neither model weights nor code have been released.
Researchers from the Hong Kong University of Science and Technology (Guangzhou) and Alibaba have introduced ReWorld to address a structural conflict in interactive world models: responding precisely to keyboard or camera actions requires attention to focus on the latest frames, while remembering locations visited long ago requires retaining the full history. With only a sliding window, older scenes are quickly evicted; retaining the entire KV cache, however, causes VRAM usage to grow continuously with the journey.
During training, ReWorld mixes per-head attention windows of different lengths. Most heads attend only to recent context, while a small subset can access long-term history. Random head routing prevents control or memory capabilities from becoming permanently dependent on specific heads, while randomly dropping historical chunks makes sparse memory part of the training distribution. At inference time, the system combines a fixed-size KV cache with a pose-indexed landmark bank: a landmark is written only after the camera has moved far enough, and nearby views are retrieved according to the current pose when the next video segment is generated. As a result, memory cost does not increase linearly with the total distance traveled.
The data pipeline aligns eight source categories—including Unreal-rendered footage, gameplay exploration, and real-world video—to a common physical scale, so the same control input represents the same travel distance. Palindromic out-and-back trajectories specifically provide supervision for returning after leaving. The model then uses distribution-matching distillation restricted to LoRA adapters to reduce diffusion sampling to four steps, allowing the same backbone to switch between a high-quality multi-step mode and a real-time streaming mode at 704×1280.
The authors report a rotational error of 11.95 degrees and the best camera-action consistency compared with six recent systems. In an out-and-back test lasting 64 seconds and covering 384 latents, a fixed 12-chunk cache could still reconstruct the starting point. However, both the videos and evaluations were provided by the team itself, and the website offers no model weights, training data, or executable code. From an engineering perspective, adoption should wait for independently measured frame rates, hardware and VRAM configurations, ablations of landmark-bank capacity, and evidence of long-term stability under free exploration, dynamic objects, and accumulated pose error.