世界模型／多模態生成
EchoWM Integrates Six-Degree-of-Freedom Navigation and Native Audio Into a World Model, but the Interactive Code Has Yet to Be Released
The JD.com Joy Future Academy team has introduced EchoWM, which uses a unified trajectory interface to control 720p audiovisual worlds from both first- and third-person perspectives. The paper describes the system as an “open” model, but the currently linked repository still provides only a long-video pipeline; EchoWM’s code and weights have yet to be released.
EchoWM’s key technical contribution is not attaching sound effects to a video generator, but enabling a single model to synchronously generate 720p visuals, ambient sound, music, and speech from continuous navigation inputs. The system converts discrete commands such as keyboard inputs and continuous camera poses into relative six-degree-of-freedom trajectories with metric scale. In first-person scenes, these trajectories move the observer; in third-person scenes, the model learns the coordinated motion of the camera, characters, and vehicles, eliminating the need to build a separate controller for each viewpoint.
The training data combines internal and online gameplay videos, Unreal Engine simulations, and general web videos, providing precise geometry and rich audio, respectively. The training process first continues audiovisual pretraining, then freezes the backbone and trains only the trajectory channel, followed by joint fine-tuning at a low learning rate. Finally, the model undergoes autoregressive post-training using histories it generates itself. A four-step causal variant, EchoWM-Flash, targets lower-latency continuous generation.
The paper reports leading or near-leading results across WBench’s 158 navigation cases and on SANA-WM-Bench, while also demonstrating multi-turn returns to previously visited scenes. These results, however, remain author-reported evaluations. The model has no explicit persistent 3D memory, so geometry, characters, state, and audio may all drift during extended operation. Controllable speed, displacement, and rotation are also limited by the range represented in the training data. More importantly, the official GitHub repository currently states that the JoyAI-Echo 1.5 code is “coming soon.” The existing inference code primarily supports text-to-long-video generation and cannot reproduce EchoWM’s interactive navigation. Engineering teams should wait for the interactive weights, end-to-end latency figures, VRAM requirements, and independent reproduction results before deciding whether it can serve as a real-time simulator.