PyTorch Closes Triton Decode Performance Regression Issue; 3.9 Branch Revert Restores Speed for Specific Attention Kernels

The PyTorch Triton decode performance regression tracking wrapped up on October 26: maintainers confirmed that the large volume of warning messages had also been resolved and closed the issue. This was a follow-up to an earlier compiler revert and its validation, not a newly discovered defect that day. For teams preparing to upgrade inference environments, the key is to verify what build is actually installed. Issue record

The problem first surfaced in an SGLang workload, and the reporter also provided a standalone reproducer that did not require model weights. The Triton patch notes say that a change to the predicate operand for masked asynchronous copies interacted with another change to global loads, affecting backend instruction scheduling. On GB200, latency for a specified GQA decode kernel rose from about 10.9 microseconds to 17.4 microseconds. Register use, spills, and numerical results were unchanged, showing that resource usage alone could not explain the regression. Patch analysis

The fix was to revert the related changes in Triton’s release/3.9.x branch; the revert was merged on October 20. The trade-off was giving up the original changes’ benefit in another pipelined-copy tutorial example, illustrating that compiler optimizations can produce different results depending on kernel shape and scheduling. Triton’s official attention tutorial also provides an executable kernel and measurements across different precisions and shapes, useful context for understanding these performance differences. Merge record, official tutorial

In a subsequent GB300 comparison, the specified case improved from 16.86 microseconds on an earlier build to 10.72 microseconds. Another participant confirmed that the previously mentioned SGLang workload regained performance on GB200 and GB300. However, the detailed public data is still mainly for synthetic kernels, and cannot be translated into a throughput gain for a complete service. A maintainer also had trouble testing the fix because the environment retained an old wheel with the same 3.9.0 version. Engineers should verify the package source and build, then retest end-to-end latency using their actual KV configuration and batch sizes. Validation and limitations