生成式影音與 3D
StateFlow Replaces Shot-by-Shot Regeneration With Persistent 3D State, Earns 4.5/5 User Rating for Video Previews
StateFlow stores object geometry, poses, semantic states, and camera settings as an editable 3D world, leaving the video model to handle visual refinement. In a 30-person study, its video previews received an overall rating of 4.5/5, though the system relies on several closed models and its code has not yet been released.
Teams from Beijing Jiaotong University, the Beijing Academy of Artificial Intelligence, and other institutions have introduced StateFlow, an attempt to address the object drift, shifting spatial relationships, and loss of camera control commonly seen when repeatedly editing text-to-video generations. Rather than treating each shot as an independent generation task, the system maintains a persistent, object-level world state. Each object includes geometry, 3D position and pose, as well as semantic attributes such as appearance and event state. When subsequent edits change the style, move an object, or damage a scene, only the affected state records are updated.
During construction, the system first generates a front view and a bird’s-eye view: the former provides object appearance, while the latter constrains the global layout. If the two images conflict over the number or positions of objects, a VLM determines which elements are hallucinations and which should be retained, then adjusts the 3D bounding boxes using collision, boundary, and semantic priors. Images of the individual objects are subsequently passed to Hunyuan3D for conversion into assets. Camera planning uses a “propose–render–refine” loop: the VLM first generates a trajectory based on the director’s intent, a low-cost renderer checks for occlusion, framing, collision, and motion problems, and the system then locally adjusts distance, viewing angle, focal point, or field of view.
The authors assembled the experimental system using Gemini 3.1, Nano Banana 2, Hunyuan3D, and Seedance 2. In the 3D scene comparison, StateFlow achieved a CLIP-I score of 0.788, above SynCity’s 0.689. Its CLIP-T score was also higher at 30.214 versus 22.880, although SynCity led on HPS V2. Thirty participants rated 12 prompts for each setting. StateFlow received an overall average score of 4.5/5 for both scenes and final videos, while the strongest video baseline, Seedance 2, scored 3.6.
The results should not yet be considered a breakthrough in general-purpose world models. The tests did not use real 3D scenes as geometric ground truth, the automated evaluator used the same Gemini 3.1 model, and the code link on the project page is still marked “soon.” The full pipeline takes several minutes and depends on multiple third-party models. From an engineering perspective, the key questions after the code is released will be whether the state format can be replayed across generators, and whether cost, failure rates, and consistency remain manageable during long editing sequences.