機器人與推論最佳化
QuantWAMs Runs World Action Models Primarily at 4-bit, Targets 29% of FP16 Memory Usage for Key Modules
For models that generate future frames and robot actions simultaneously, QuantWAMs turns quantization calibration into a three-level decision process covering structure, joint objectives, and closed-loop trajectories. The simulated success rates of two models were within 0.2 to 0.7 percentage points of FP16, but the speed measurements cover only specific compute modules, while each real-robot comparison involved just ten trials per task.
World Action Models (WAMs) combine a video generator and an action policy within the same iterative denoising process, allowing a robot to generate control commands while predicting future frames. The challenge with this design is not merely model size: every action changes the input to the next iteration, so early quantization errors may accumulate along a closed-loop trajectory. As a result, conventional post-training quantization (PTQ) calibrated using single-image reconstruction error or open-loop outputs may not preserve actual task success rates.
QuantWAMs divides quantization decisions into three parts. First, it merges activation statistics only across modules with compatible coordinate systems, uses smoothing and Hadamard rotation to handle outlier channels, and retains a small number of important channels in BF16. Second, the system calculates empirical-Fisher salience from the joint gradients of video and action outputs. It then decides which linear layers should be upgraded from NVFP4 W4A4 to FP8 W8A8 at whole-layer granularity, rather than using unstable fine-grained masks. Finally, it replays fixed interventions using closed-loop states actually reached by the FP16 model, then reschedules which denoising steps require higher precision without increasing the overall precision budget.
The team tested QuantWAMs on Fast-WAM and LingBot-VA across RoboTwin 2.0, LIBERO, and a physical AgiBot G2 robot. Configurations operating primarily in W4A4 achieved average simulated success rates only 0.2 to 0.7 percentage points below FP16. For the targeted video and action blocks, weights plus peak activation memory were approximately 29% of FP16 levels, while block-level speed increased by 1.4× to 1.6×. Calibration, trajectory analysis, scheduling validation, and final testing used mutually separate datasets, reducing the risk of directly tuning against the test set.
However, these figures should not be interpreted as making the entire robot 1.6× faster: the measurements excluded embeddings, projections, the VAE, and the rest of the control pipeline. The three real-robot tasks had only ten trials each. QuantWAMs achieved an average success rate of 56.7%, compared with 63.3% for FP16, leaving too few samples to establish equivalence. At this stage, the results are better viewed as evidence of feasibility. The next questions are whether the code and kernels will be fully released, whether the approach can be ported to non-Blackwell GPUs, and whether calibration on one set of tasks can still generalize to unseen manipulation tasks.