推論系統
PhyAI Unifies Cloud and Edge Inference for Robot Models, Accelerating Four VLA Models by Up to 4.65×
The open-source PhyAI inference engine uses model adapters to isolate architectural differences, allowing a single execution environment to cover cloud rollouts, GPU serving, and onboard deployment. The authors measured speedups of 1.40× to 4.65× over official implementations, though some specialized runtimes remain faster.
Robotics teams typically maintain separate inference programs for training rollouts, offline evaluation, edge GPUs, and onboard control, even when they use the same weights and action semantics. Submitted on August 4, PhyAI attempts to consolidate these paths into a latency-first runtime: model-specific conditioning inputs, solvers, caching, and output logic remain in adapters, while graph execution, kernels, memory management, and distributed serving are shared.
The library currently supports vision-language-action models including π0, π0.5, GR00T N1.7, and MiniCPM-Robot, as well as the Cosmos3 family of world-action models. It can run on Jetson, a single GPU, or a cluster, and supports data parallelism, tensor parallelism, and classifier-free guidance parallelism. The project uses kernels such as FlashInfer and Humming. Its repository also lists W4A8, W8A8, and W8A16 quantization paths, some of which are still under review.
The authors report that PhyAI runs four VLA models 1.40× to 4.65× as fast as their official implementations. For Cosmos3-Nano-Policy-DROID on eight H20 GPUs with CFG=2 and TP=4, latency fell from 2.46 seconds to 1.18 seconds. Profiling also shows why FLOPs alone cannot identify bottlenecks: on Hopper at batch size 1, π0.5’s action expert accounts for only 8.8% of FLOPs but 57.2% of latency. At batch size 32, its share of latency falls to 13.5%.
The significance of this work extends beyond isolated speedups: it reduces execution differences when models move from research environments to robot hardware. The team proposes a control-time Roofline model to distinguish whether a control loop is constrained by model inference or the physical environment. In its LIBERO experiments, π0.5 is already environment-bound, while Cosmos3 remains inference-bound. Engineers should next watch the progress of quantization merges, support for non-NVIDIA backends, jitter and tail latency on real robots, and whether the paper’s results can be independently reproduced beyond the official baselines.