推論系統／實體 AI
FlashDrive Combines Caching, Speculative Decoding, and W4A8 to Cut 10B Autonomous-Driving VLA Latency to 151 Milliseconds
FlashDrive tackles four bottlenecks—visual encoding, LLM prefill, reasoning tokens, and trajectory denoising—cutting Alpamayo 1.5-10B single-window latency on an RTX PRO 6000 from 717 to 151 milliseconds. The open-source implementation approaches real-time inference, but requires specialized fine-tuning, and its closed-loop evaluation covers only 100 simulated clips.
Rather than optimizing yet another individual operator in a VLA model, FlashDrive divides autonomous-driving inference into four stages and optimizes each separately. For continuous camera frames, it preserves the KV cache along the temporal axis, rerunning visual encoding and prefill only for the newest frame. For low-entropy driving-reasoning text with strong block-level correlations, it uses a non-autoregressive diffusion model to draft eight tokens at once, which are then verified by the original model. The final trajectory generator uses flow matching. The team observed that the velocity field changes more sharply at the beginning and end and remains flatter in the middle, so it recomputes the action expert only at selected denoising steps.
At the system level, FlashDrive also layers in CUDA Graphs, fused QKV and MLP projections, and ParoQuant W4A8. The VLM backbone runs with INT4 weights and INT8 activations, while the numerically sensitive action expert remains in BF16, reducing memory usage to approximately 18.3 GB. Using a single RTX PRO 6000 and 100 clips from the PhysicalAI-AV dataset, the researchers cut Alpamayo 1.5-10B latency from 717 milliseconds to 151 milliseconds, increasing throughput from roughly 1.4 Hz to 6.6 Hz. The six-trajectory configuration achieves greater speedups on some GPUs, but those results are not directly comparable with the single-trajectory figures.
In open-loop evaluation, `[email protected]` improved from 1.705 meters to 1.573 meters, while six-sample `minADE6` worsened from 0.767 meters to 0.844 meters. In AlpaSim closed-loop tests, collision and off-road rates decreased, but maximum trajectory distance and wrong-lane rate deteriorated, so the results do not support a claim of improved overall safety. From an engineering perspective, streaming KV reuse requires fine-tuning on approximately 600,000 samples, and the draft model also requires additional training; this is not simply a matter of swapping the runtime under existing weights. The code and derived checkpoints have been released, but the Alpamayo weights retain their noncommercial license. Real-vehicle latency, sensor jitter, and long-term cache drift remain to be validated.