4DAnyone Releases Model and Inference Code, Expanding Monocular Human Videos into Up to 48 Reconstructed Views

Teams from Zhejiang University, Robbyant, Ant Group, and other organizations have released 4DAnyone’s Apache 2.0-licensed inference code and model assets on Hugging Face. The system lets developers generate videos from multiple fixed viewpoints using an uncalibrated monocular human video, then pass them to 4D Gaussian Splatting (4DGS) to reconstruct a freely viewable dynamic human. The code supports a six-view configuration for quick testing, 24 views for full 360-degree coverage, and a 48-view configuration spanning three elevation levels.

The core challenge is that a video diffusion Transformer cannot accommodate the dozens of views required for reconstruction in a single forward pass. If the views are split into groups and denoised independently, the groups cannot exchange information, causing drift in body proportions, clothing, and occlusion relationships. If every previously generated result is used as a reference, the context grows with the number of views at O(N). 4DAnyone’s Reference Context Packing compresses reference views at different resolutions into a fixed-length context, reducing this component’s complexity to O(1). Target Context Routing rotates the views assigned to each group during high-noise stages, allowing global structure to propagate across groups. During low-noise stages, it fixes groups of adjacent views to focus on stabilizing details.

For geometric conditioning, the system avoids dense depth, which is susceptible to monocular estimation errors. It first recovers human motion with GVHMR, then uses z-buffering to render a compact set of 40 3D skeletal keypoints into the target views, preserving front-to-back limb occlusions. Facial and finger details are left to the source video and generative model. The training data combines the team’s MVGameHuman dataset, light-field studio captures, and in-the-wild videos. The paper reports results on DNA-Rendering and DyMVHumans that outperform existing novel-view video and downstream 4DGS methods, although the findings still come primarily from the authors’ own evaluations.

The current reproduction requirements remain substantial. Recommended inputs are at least 720p and 121 frames, feature one person in portrait orientation, and contain only slight camera movement. Inference with less than 32GB of memory, TensorRT and sparse-attention acceleration, and a fully open-source 4DGS reconstruction pipeline remain on the roadmap. For now, the release is best suited to evaluating the multiview observation generation stage and should not yet be considered a one-click production pipeline that turns arbitrary smartphone footage into a complete 4D asset.