HelixWorld Releases HelixBench to Evaluate Whether Generated Soundscapes Follow Viewpoint Changes

On October 12, Noiz AI and the Hong Kong University of Science and Technology’s HelixWorld project announced the release of HelixBench, extending interactive world model evaluation to whether sound changes with the viewpoint. This update makes evaluation resources public: preview model weights and inference code were released on September 3, and the paper went online on September 29. Project release history

The benchmark targets a specific gap: when a generated scene turns, the position of a sound source in the left and right audio channels should change too. A workflow that adds audio to a video afterward may not account for the user’s actions and camera trajectory. HelixWorld uses a joint audio-video generation architecture, with cross-modal attention linking visual geometry and audio features; HelixBench provides a way to test this coupling. Technical announcement

The public package includes 1,015 starting images, audio-video descriptions, evaluation code, and fixed reference features. For each sample, users must provide a generated MP4 and record the sample ID, absolute path, and SHA-256 in a manifest. The six metrics are reported separately, with no aggregate score: KL and FAD assess audio distribution and quality, while ImageBind and CLAP measure cross-modal and text alignment. DeSync uses 165 event cases, and Spatial uses 150 viewpoint cases. Dataset documentation

The current Spatial v3 protocol particularly affects how results should be interpreted: invalid clips score −100, eligible clips with a sound source only at the center score zero, and all others are scored on directional correctness in lateral views. Before comparing models, confirm that they use the same protocol, and examine audio quality, synchronization, and viewpoint consistency separately so a single number does not obscure failure cases. The installer pins third-party package versions and checks downloaded weights; the project says the full workflow has been tested on a single NVIDIA A10G. Evaluation protocol

The limitations are clear: the public package does not include source or generated videos, only audio-video descriptions and reference features, so researchers cannot inspect every reference audio-video pair directly from the package. The model’s real-time performance was measured on an H800 and should not be assumed to apply to consumer GPUs. Next, watch for independent reproductions, scoring-version changes, and error patterns during long turns or when sound sources leave the frame. Dataset, performance conditions