vLLM Community Reports Output Divergence After Restart; Core Autotuning May Affect Reproducibility

On September 27, the vLLM community reported that restarting a process could change its output even with a fixed random seed and temperature set to zero. The case used vLLM 0.30.0, PyTorch 2.13.0, Qwen3-8B, and an RTX 6000 Ada. Processes that selected different kernel configurations produced different token sequences for three of eight prompts. [Issue report](https://github.com/vllm-project/vllm/issues/58899)

The reporter traced the cause to a Triton kernel that fuses Q/K normalization with rotary positional encoding. Based on measured speed, the compiler selects a configuration with a reduction block size of either 64 or 128. The two configurations sum values in a different order, and floating-point mantissa differences may change greedy decoding results. Forcing a specific candidate configuration made the corresponding output reproducible. Differences across processes were still observed with `VLLM_BATCH_INVARIANT=1` enabled. [Reproduction steps](https://github.com/vllm-project/vllm/issues/58899)

The official documentation has never guaranteed reproducibility with the default configuration. It lists batch invariance as a way to obtain consistent results in online serving, provided the hardware and vLLM version are the same. A fixed seed controls the source of randomness; in this case, if the underlying numerical computation path differs, saving sampling parameters alone is not enough to establish a reliable baseline for comparison. [Reproducibility documentation](https://docs.vllm.ai/en/latest/usage/reproducibility/)

A related issue involves PyTorch: when restoring global state after Dynamo tracing ends, the process may also overwrite Inductor's determinism setting. A patch proposed on September 25 changes this to restore the underlying global flag directly, preserving the compiler's separate setting. The proposal includes a regression test using CPU tensors and the eager backend to isolate the state restoration issue, but GPU serving behavior still needs to be validated separately. As of this review, the proposal has not been merged and cannot be considered a resolution to the vLLM case above. [Patch proposal](https://github.com/pytorch/pytorch/pull/198572)

Batch invariance is still experimental. The latest documentation also lists some CUDA configurations that can bypass `torch.compile`, showing that the actual execution path varies with configuration. A report from a single version should not be generalized to every deployment. [Feature limitations and implementation](https://docs.vllm.ai/en/latest/features/batch_invariance/)

From an engineering perspective, this means evaluations and regression tests should cover full process restarts and preserve the model revision, package versions, and compilation settings, while comparing token sequences and log probabilities. Investigations should also distinguish stability within a single process from consistency across launches, to avoid mistaking infrastructure differences for effects caused by prompt or weight changes. Follow-up should track maintainer confirmation, patch merging, and reproduction across hardware. The current evidence comes mainly from the reporter and is not enough to estimate how often this occurs or its effect on model quality.