GitHub Repo
vLLM Community Investigates NVFP4 Cache Layout Gap; Interleaved Storage May Overlap Data and Scale Values
vLLM community reports from October 12–13 indicate that NVFP4 KV cache reads and writes may be misaligned under certain token-major layouts. Another participant reproduced data overlap at the kernel level. Maintainers have yet to clarify the supported configurations, so the issue should not be assumed to affect all production deployments.
The vLLM community reported an NVFP4 KV cache anomaly on October 12, and another participant added an independent kernel test the following day. The discussion centers on the token-major physical layout in BLNHC: when K, V, and quantization scale data are stored in an interleaved arrangement, the write and read paths may disagree about byte positions. The original report said that a particular QSA model deployment still returned tokens, but its answers became incoherent. Community report and reproduction
KV cache quantization reduces the memory cost of storing attention state during inference, allowing more tokens to fit. This optimization also requires numerical data to remain correctly paired with its scale factors. The vLLM NVFP4 helper-function documentation specifies that data and scale values must each be packed contiguously within each K or V region; callers must first split the two sides out of the combined cache. Therefore, matching tensor shapes alone does not prove that the physical storage layout is compatible. Quantized cache documentation, NVFP4 API documentation
On October 13, a participant reproduced errors in an NVFP4 write-then-read test on an RTX 5090 without loading a model. They noted that the issue may extend beyond write-side stride calculations: the data and scale views created by the read helper may themselves point to the same byte. This suggests that changing only the write path may not fix the interleaved layout. The test provides evidence about the kernel and storage contract, but does not validate full model serving. Test and limitations
Key questions about the scope remain unresolved. A participant noted that the related QSA NVFP4 integration proposal has not been merged, and the QSA type list on the main branch does not declare NVFP4 support. Engineers should first check the actual integration path, then round-trip layout test data produced by the configurator. They should monitor how maintainers define the support contract and whether initialization will reject incompatible layouts. Current reports are insufficient to characterize this as a generation defect affecting all NVFP4 models. Discussion of support scope