電腦視覺／邊緣推論
Ultralytics 8.4.143/8.4.144 Integrates YOLO26 INT8 QAT Across Training, Checkpointing, and TensorRT Export
YOLO26 can now run quantization-aware training with `quantize=8`, save quantization ranges in checkpoints, and directly export ONNX models with Q/DQ nodes or TensorRT engines. The subsequent 8.4.144 release adds guidance on quantization gains and numerical stability fixes, while also highlighting the deployment path’s clear limitations.
Ultralytics added native INT8 quantization-aware training (QAT) for YOLO26 in version 8.4.143. Developers can continue using the existing training entry point and simply set `quantize=8`. The system then uses NVIDIA ModelOpt to insert fake-quantization operations, allowing the weights to adapt during fine-tuning to INT8 clipping and scaling errors instead of applying post-training quantization (PTQ) with calibration data only after training is complete.
The significance of this change extends beyond a new training parameter. Quantizer state and calibration ranges are saved in checkpoints alongside standard model layers and reconstructed when training resumes. The exporter recognizes QAT models, skips additional PTQ and calibration, and directly generates ONNX models containing Quantize/Dequantize (Q/DQ) nodes or passes them to TensorRT to build INT8 engines. The output head remains in floating point for now, and QAT models cannot use `torch.compile`. Loading checkpoints still requires `nvidia-modelopt`, and officially supported export formats are limited to ONNX and TensorRT engine.
Released within a day, version 8.4.144 further fixes mask dtypes for empty-label batches, FP16/BF16 numerical boundaries, Triton input precision, and custom YAML loading precedence. It also adds a comparison between QAT and calibration-based PTQ. The official conclusion is that the additional benefit is limited for smaller YOLO26 models; larger models, or cases where PTQ causes a substantial accuracy drop, are more likely to justify the cost of fine-tuning again.
Engineering teams should remeasure mAP, latency, and power consumption using their actual TensorRT version, GPU, and data distribution, as current quantization results primarily come from the project’s own COCO tests. Before merging, the change also exposed issues involving duplicate normalization of classification inputs, distributed calibration synchronization, and quantizers being placed on the wrong device when interrupted training was resumed. Although the maintainers say these issues have been fixed and validated on real hardware paths, production testing should still specifically cover multi-GPU training resumption and regression testing of exported models.