推論系統
NVIDIA Dynamo Prewarms a Standby Engine with Shared HBM Weights, Cutting Failover from 283 Seconds to 7.3 Seconds
Dynamo’s experimental Shadow Engine Failover lets two inference processes map the same GPU weights, avoiding a model reload after a process crashes. NVIDIA’s GLM-5.2 test showed a nearly 39× reduction in recovery time, but the feature currently primarily supports vLLM and cannot handle GPU, node, or cross-node failures.