GitHub Repo
PyTorch Community Reports GB10 Source Build Defect That May Invalidate CUDA Context During FP8 Matrix Operations
On November 9, the PyTorch community reported that GB10 row-wise scaled FP8 matrix operations may trigger a GPU error when building only for the 12.1 architecture. The author proposed a compile-condition fix and reported passing tests, but said official packages are unaffected; the fix’s release status remains unconfirmed.
On November 9, the PyTorch community reported an issue affecting source builds for NVIDIA GB10/DGX Spark: after setting TORCH_CUDA_ARCH_LIST=12.1, the first execution of row-wise scaled FP8 torch._scaled_mm may trigger a device-side trap, invalidating the CUDA context and causing subsequent CUDA calls in the same process to fail. The author said official release wheels are unaffected because they include a different architecture build path. Issue report
The key distinction is between hardware capability and the compilation target. NVIDIA lists GB10 as compute capability 12.1; CUDA documentation explains that using architecture-specific instructions requires selecting a compile target with an a suffix. Specifying the base compute capability alone does not enable the full instruction set. This means that “supports this GPU” and “this specific kernel was compiled correctly” are two separate conditions to verify. GPU compute capability table, CUDA architecture target documentation
According to the author’s analysis, cmake/Codegen.cmake originally added sm_121a for the relevant files only when the existing flags included compute_120; specifying only 12.1 therefore missed this step. The proposed fix is to make the condition recognize both compute_120 and compute_121. The author reported that, after making the change on the same machine, the related matrix operation tests passed: 543 passed and zero failed. This result was provided by the submitter and does not yet constitute cross-environment validation. Build analysis and test results
For engineers who compile low-precision inference environments themselves, this report is a reminder to run actual FP8 kernel tests after a successful build and retain the compile flags for comparison. The available evidence is limited to a specific GB10, CUDA 13.0, and development environment; it should not be interpreted as affecting all Blackwell systems or PyTorch installations. Follow-up questions include whether the fix will be merged, which release will include it, and whether single-architecture builds will receive regression coverage.