Ultralytics 8.4.157 adds CUDA Graph replay, cutting YOLO26n FP16 test latency by about 20%

Ultralytics released version 8.4.157 on September 20, improving the TensorRT export and execution paths for YOLO models. The package is also available on PyPI. The gains come from changes to precision calibration and kernel launch overhead. Users can keep their existing model weights but will need to benchmark their deployment pipelines again. [Release notes](https://github.com/ultralytics/ultralytics/releases/tag/v8.4.157), [package listing](https://pypi.org/project/ultralytics/8.4.157/).

The first change concerns mixed-precision conversion in TensorRT 11. Previously, calibration used random noise, which could inflate activation values in early layers and cause some convolutions to remain in FP32. The new version uses bundled real images, allowing operations whose values fall within range to be converted to FP16. Engines must be rebuilt to benefit from this export improvement. [Export code](https://github.com/ultralytics/ultralytics/blob/v8.4.157/ultralytics/utils/export/engine.py).

The second change captures a CUDA Graph when an eligible fixed-shape engine is loaded. Subsequent calls copy inputs into a fixed buffer and replay the graph, reducing the overhead of launching kernels individually. Engines with dynamic shapes, DLA, or embedded non-maximum suppression do not use this path. If an execution submission is rejected, the implementation also falls back, avoiding replay of an empty graph that would leave stale outputs in place. For vision services using fixed input sizes and small batches, this provides a concrete way to reduce scheduling overhead. Requests with varying input sizes still require a separate batching strategy. [Execution code](https://github.com/ultralytics/ultralytics/blob/v8.4.157/ultralytics/nn/backends/tensorrt.py).

Maintainers tested with an RTX PRO 6000 Blackwell, TensorRT 11.3, and 640-pixel inputs. YOLO26n's batch-1 FP16 engine latency fell from 0.526 to 0.421 milliseconds, a reduction of about 20%. YOLO26s saw roughly an 18% reduction with batch-1 INT8, while INT8 at a batch size of eight showed almost no improvement. The measurements cover only engine calls and device synchronization, and accuracy checks used the small coco128 dataset. These results cannot be taken as a direct measure of full-service throughput. [Final tests](https://github.com/ultralytics/ultralytics/pull/26223).

The release notes still list a SiLU activation fusion rewrite, but it was withdrawn during final review. Maintainers identified a risk of miscompilation in the relevant fusion and noted that tests comparing only classification scores could miss bounding-box coordinate errors. Assessments of this update should therefore be based on the changes actually merged. [Review record](https://github.com/ultralytics/ultralytics/pull/26223).

Engineering evaluations should hold hardware, batch size, and input dimensions constant, then compare latency after warmup, accuracy on the full dataset, and end-to-end processing time. The official documentation also notes that INT8 results depend on calibration data and hardware; lower precision does not guarantee faster execution for every workload. In a real-time camera inference service, decoding, resizing, data transfers, and postprocessing may still dominate processing time. Saving one-tenth of a millisecond in the engine does not necessarily produce a proportional reduction in total processing time per image. Deployment teams can retain build configurations from before and after the update and record engine and application benchmarks separately. [Deployment documentation](https://docs.ultralytics.com/integrations/tensorrt).