FlashDrive Jointly Trims Four-Stage VLA Inference, Cutting Alpamayo 1.5 Single-GPU Latency to 151 ms

Latency in autonomous-driving Vision-Language-Action models is not concentrated in a single core component. Instead, it is distributed across visual encoding, language-model prefill, reasoning-token decoding, and flow-matching-based action generation. [FlashDrive](https://arxiv.org/abs/2608.12932) therefore adopts a joint algorithm–system design that removes a different form of redundant computation from each of the four stages, rather than merely replacing the attention kernel.

The first optimization exploits temporal overlap in consecutive multi-camera inputs. In a four-frame sliding window, three frames typically remain unchanged on each update, so the system encodes only the new frame, stores pre-RoPE keys, and applies new rotary position embeddings when positions shift. Combined with a custom mask, this reduces the effective sequence length for visual processing and prefill by approximately 75%. Because old KV entries were computed under different contexts, directly reusing them causes distribution shift. The authors freeze the VLM and fine-tune only the action expert, which is more sensitive to the resulting errors, to restore trajectory accuracy.

During reasoning, the block diffusion model DFlash drafts an entire sequence of low-entropy, fixed-format driving reasoning tokens in one pass, after which the original model verifies them. On the action side, the authors observe that the flow-matching velocity field changes in a U-shaped pattern: changes are large in the first and last steps, while cosine similarity exceeds 0.99 in the middle. The system therefore recomputes only the steps near the endpoints and caches the intermediate velocities. It then layers on W4A8 ParoQuant, CUDA Graphs, and QKV and MLP kernel fusion.

In the full paper, FlashDrive reduces Alpamayo 1.5-10B latency on an RTX PRO 6000 from 717 ms to 151 ms, increasing throughput from 1.4 Hz to 6.6 Hz; [email protected] changes by only 0.08 meters. An earlier [project page](https://z-lab.ai/projects/flashdrive/) reported 159 ms, indicating that the published figures are still shifting with different versions or test configurations. More importantly, the code and weights have not yet been released, and the closed-loop results come from a simulated environment. A rate of 6.6 Hz alone also does not demonstrate compliance with real-vehicle control, safety-redundancy, or worst-case latency requirements. Engineering teams should next watch for reproducible implementations, cache drift over long sequences, and power consumption and tail latency on Jetson Thor.