vLLM 0.32.0 Adds Structured Decision Endpoint; Check Python and Streaming Compatibility Before Upgrading

vLLM released version 0.32.0 on October 18, 2026, matching the date in the GitHub release record and the PyPI package listing. The release adds POST /v1/systemone, which lets applications submit a state and typed questions and receive probabilities for each option. These results can be integrated into classification, routing, and scoring workflows. Release notes, PyPI package

For supported generative models, the endpoint processes each question separately, reads the log probabilities of labels at the first response token, and applies softmax to calculate the option distribution. Questions support multiple choice, yes/no, and ordered scoring. With prefix caching enabled, prefill results for shared state can be reused. From an engineering perspective, this could reduce a classification workflow’s reliance on parsing free text. However, the endpoint being enabled by default does not mean every model supports it; requests return 501 when the required reading strategy is unavailable. Version documentation

The probability fields also have limitations. A high probability after normalization across options does not necessarily mean the model originally favored producing a valid label. The documentation also provides label_mass, the total probability mass assigned to labels across the entire vocabulary, and confidence, which combines that total with the share assigned to the selected answer. Engineers should monitor these values together and validate thresholds using their own data. Probability definitions

Reproducibility also requires testing. According to the project, probabilities for the same request can vary with batch composition and prefix-cache hits. Setting VLLM_BATCH_INVARIANT=1 provides reproducible readings, at the cost of throughput. Online routing systems that use small probability differences to choose the next step should measure whether batch variation can flip decisions. Execution limitations

For upgrades, 0.32.0 drops Python 3.10 support, tightens the default CORS origins, and renames reasoning stream events in the Responses API. Operators should check their base images, browser origin settings, and event parsers. Whether the new endpoint improves real-world cost and latency still needs to be measured with the target model, question count, and cache conditions; those gains cannot be inferred from the feature alone. Compatibility changes