ONNX Runtime 1.31.0 Stabilizes Model Package APIs and Strengthens Quantized MoE Inference and Plugin Compatibility Requirements

Microsoft released ONNX Runtime 1.31.0 on October 9, with updates covering model deployment interfaces, quantized inference, and execution provider integration. PyPI made the new package and multi-platform wheels available the same day, confirming that installable release artifacts are available. Release notes, PyPI package

The most direct change for deployment is that the model package and EPContext data read/write callbacks have become stable C/C++ APIs. Model packages can be used to select compatible model variants and create inference sessions; EPContext callbacks let applications manage the storage of backend context data. Integrations that used the experimental interface need to migrate, with the model package entry point changing to OrtApi::GetModelPackageApi. From an engineering perspective, this helps bring model selection and storage integration under a more stable API contract. API updates

On the CPU path, prepacking of eligible weights is now parallelized, while MLAS QNBit kernels and grouped expert dispatch accelerate blockwise INT4/INT8 MoE. These changes target initialization and expert matrix computation costs, respectively, but the release information alone does not establish latency improvements for any specific model. Teams should measure performance with their own batch sizes, quantization settings, and hardware. CPU updates

When upgrading, check the combinations of the core runtime and GPU plugins carefully. The official release lists CUDA Plugin EP 0.3.0 and WebGPU Plugin EP 0.5.0 as the corresponding versions, and requires both sides to use the same contributed-operator schemas. Schema mismatches can cause incorrect execution or crashes, and may not be detected during plugin registration; the official recommendation is therefore to build from the same source revision. Compatibility requirements

Another behavior change is that GatherBlockQuantized now returns a zero slice for out-of-bounds indices, which differs from the error semantics of ONNX Gather. Engineering teams should first verify API migrations and plugin versions, then use existing models to validate output consistency, cold-start time, and throughput. Keep tests for out-of-bounds indices in place to avoid letting the new behavior mask upstream data problems. Behavior change