vLLM 0.30.0 adds resident GPU weight caching for reuse across engine restarts

vLLM released version 0.30.0 on September 22, adding the Fast Start weight caching mechanism. It moves the lifecycle of model weights outside the inference engine: a separate daemon for each GPU retains weights that have already undergone quantization processing and tensor-parallel sharding, allowing the engine to reconnect to them after a restart. The release notes also list support for FP4 checkpoints and multi-node tensor parallelism. [Release notes](https://github.com/vllm-project/vllm/releases/tag/v0.30.0)

The design addresses the cost of repeatedly loading weights when services restart. The implementation passes CUDA IPC handles through a local Unix socket, allowing a new engine to map existing GPU memory. It can be enabled by specifying `--load-format ipc_cache` at startup. For deployments that repeatedly adjust service settings while retaining the same model weights, this offers a way to reduce disk reads. [Implementation proposal](https://github.com/vllm-project/vllm/pull/54921)

The loader first creates a model skeleton on the meta device, then replaces its parameters and buffers with cached tensors, skipping post-load weight processing. The default `zero_copy` mode shares the daemon’s GPU memory; `copy` mode copies the weights into the engine’s own memory and then requests that the cache be released. These modes differ in memory ownership, so failure recovery and GPU memory planning must be validated separately for each. [Loader documentation](https://docs.vllm.ai/en/v0.30.0/api/vllm/model_executor/model_loader/weight_cache/ipc_loader/)

Supported configurations still have limits. The daemon supports tensor and expert parallelism but rejects pipeline and data parallelism at startup. Multi-node deployments must launch daemons on each node and coordinate the global tensor-parallel group. IPC handles remain local to each node, with each engine worker obtaining the weight shard for its local GPU. [Deployment documentation](https://docs.vllm.ai/en/v0.30.0/api/vllm/model_executor/model_loader/weight_cache/daemon/)

If the cache daemon cannot be reached or the configuration fingerprint does not match, the default behavior is to fall back to loading from disk. A successful service startup therefore does not necessarily indicate a cache hit. Zero-copy mode also cannot be used with sleep mode that offloads weights through CuMemAllocator, so existing GPU memory conservation workflows need to be checked again. [Compatibility notes](https://docs.vllm.ai/en/v0.30.0/api/vllm/model_executor/model_loader/weight_cache/ipc_loader/)

Based on the architecture, the benefits likely depend on whether weights can remain on the same GPUs and how much of the original startup time was spent loading them. Engineering teams should separately measure cache hits, disk fallback, and daemon restarts, recording time to service readiness, first-request latency, and peak GPU memory usage. Beyond model weights, the service must still initialize runtime state for scheduling, communication, and other operations. Measurements should cover the entire initialization process to determine whether operational downtime is reduced. The announcement provides no broadly applicable Fast Start speedup factor, nor does it establish that steady-state generation throughput will improve.