vLLM Publishes Multi-GPU NVDEC Tests, Showing More Than Twice the Video Captioning Throughput of CPU Decoding

On September 18, vLLM published deployment test results for multi-GPU video captioning: using PyNvVideoCodec to access NVIDIA’s NVDEC hardware decoder, eight H100 GPUs with one serving replica per GPU delivered more than twice the throughput of CPU decoding. The publication covers scaling tests and deployment methods; the article’s date should not be taken as the date the feature first entered the codebase. [Engineering article](https://vllm.ai/blog/2026-09-18-pynvvideocodec)

These workloads often use vision-language models to generate short captions of around 100 to 200 tokens. The model produces output quickly, while video decoding consumes a substantial amount of time. As more GPU replicas are added on the same host, decoding can exhaust CPU capacity first. The gains therefore come from removing a data preparation bottleneck and do not directly imply a twofold speedup in model computation itself. [Test scenario](https://vllm.ai/blog/2026-09-18-pynvvideocodec)

The implementation still transfers data between the GPU and the host. The source code shows that decoded frames are copied into pinned host memory before being passed to multimodal preprocessing. It also retains decoder slots, prioritizing the reconfiguration of existing objects when switching videos to reduce the cost of repeatedly creating parsers and buffers. This path is not yet a zero-copy pipeline that keeps data on the GPU throughout. [Decoder implementation](https://github.com/vllm-project/vllm/blob/main/vllm/multimodal/video.py)

Deployers can specify `backend=pynvvideocodec` in the media parameters, but must first enable CUDA MPS because the API decoding processes and model engine share the GPU. The documentation also requires a positive value for `--mm-ipc-gpu-memory-gb`. This budget reduces the space available for the KV cache, and decoding must wait when the allocation is exhausted. Each API process reserves two hardware decoder slots by default; adding slots also increases the GPU memory reservation. [Deployment documentation](https://docs.vllm.ai/en/latest/features/multimodal_inputs/)

The next engineering step should be to hold the video format, number of sampled frames, and output length constant, then compare throughput, CPU utilization, and tail latency when scaling from one GPU to eight. The official figures come from a specific caption-generation workload, and no independent reproduction results were obtained for this report. Services with long outputs, large models, or an already full KV cache may see different gains. The documentation also recommends the DeepStream backend for streaming video, so the file-decoding configuration tested here cannot be applied directly. [Scope of applicability](https://docs.vllm.ai/en/latest/features/multimodal_inputs/)