vLLM Reportedly Terminates Engine After External KV Cache Read Failure; Scope Remains Unclear

The vLLM community received an external KV cache failure report on September 21: when running a hybrid-attention model with LMCacheMPConnector, a cache read failure was followed by an assertion error that terminated the engine process. The report includes environment details, reproduction steps, and error logs. It remains open, with no linked proposed fix or public confirmation from maintainers. [Issue report](https://github.com/vllm-project/vllm/issues/57938)

The reported setup uses an AMD MI325X and FP8 weights for Gemma 4 31B. The model combines sliding-window and global attention, with six KV cache groups configured. Reproduction involves reading through the GDS/hipFile backend and triggering a storage error. Although the written summary lists version 0.29.1, the environment output identifies the development build `0.29.1rc1.dev47+gdc36fcce9`. This does not establish that the stable release is broadly affected. [Environment and logs](https://github.com/vllm-project/vllm/issues/57938)

The key issue is the scope of the failure. The setup uses `kv_load_failure_policy=fail`, which the official documentation defines as ending the affected request with an error. The alternative, `recompute`, reschedules failed blocks for recomputation. Although the reporter wanted recomputation, the anomaly most directly supported by the available evidence is that a request failure escalated into engine termination. Simply changing the policy to `recompute` cannot yet be considered a verified solution. [Failure policy documentation](https://docs.vllm.ai/en/latest/api/vllm/config/kv_transfer/)

The logs show that the scheduler first recorded a failed request. Then, while updating cache transfer completion status, an assertion checking whether the request still existed failed. The code shown in the current documentation retains this check. That identifies a place to investigate, but it does not establish where the request was removed earlier or which component needs to be fixed. [Scheduler code](https://docs.vllm.ai/en/latest/api/vllm/v1/core/sched/scheduler/)

This type of failure warrants attention from deployment teams because LMCache’s multiprocess architecture separates the cache into a service that multiple inference instances on the same node can share, with cache resources configured independently. This architecture suggests that, alongside process isolation, the inference side must correctly handle remote errors to limit the scope of service disruptions. [Architecture documentation](https://docs.lmcache.ai/mp/index.html)

The next step should be to inject read failures in an isolated test environment, verify whether other requests continue to be served, and trace the ordering of request cancellation and transfer completion events. The available material provides no failure rate, reproduction across hardware platforms, or results from an official fix. Engineering teams should pin the exact commit used and await a minimal reproduction test and a maintainer assessment of the affected scope. Acceptance testing should also record individual request errors, process restarts, and latency for other requests separately, so that a recovered health check alone is not taken as proof that the failure was contained.