GitHub Repo
PyTorch Community Reports CUDA Batched SVD Performance Gap; Matrices Over 32 Dimensions May Run One at a Time
A PyTorch community report on October 10 says CUDA batched SVD may slow sharply when matrix size increases from 32 to 33, possibly because execution switches paths. The report includes measurements across environments and a proposed fix, but the precision trade-offs of an alternative driver and the status of an official fix still need confirmation.
On October 10, the PyTorch community filed a CUDA linear algebra performance report: when processing 50 matrices in one batch, the default path for torch.linalg.svd may suddenly take much longer as dimensions increase from 32×32 to 33×33. svdvals and pinv show similar behavior. This is a newly filed issue with a reproduction script, not an official release fix. Original report
The reporter used CUDA Events on a Tesla T4 with a PyTorch development build and reported the median of seven runs. For float32 SVD, runtime rose from about 1.10 ms to 104.8 ms. The same 33×33 input took about 3.80 ms with driver="gesvda". The report also lists similar results on a stable release and an RTX 3050, but these figures are the submitter’s measurements and cannot be used to directly estimate speedups on other GPUs or across an entire training workflow. Measurements and environment
The report points to a backend branch as the cause: when neither matrix dimension exceeds 32, PyTorch uses gesvdjBatched; above that threshold, it switches to calling gesvdj for each matrix, causing the batch to lose its execution advantage. Contributors in the discussion have proposed a fix that would make the default selection account for batch size and matrix dimensions. At the time of review, however, the issue was still open, and no released version containing a fix could be confirmed. Technical discussion
Alternative algorithms also involve trade-offs. The official documentation describes gesvda as an approximate method and notes that driver selection requires balancing speed and accuracy. For ill-conditioned matrices or cases with higher precision requirements, it recommends considering the QR-based method gesvd. The documentation also notes that when computing gradients through singular vectors, repeated or closely spaced singular values can cause numerical instability, so forward-pass runtime alone is not enough to determine whether to switch. SVD documentation
For teams that frequently decompose small matrices in low-rank analysis or research workflows, this report offers a concrete lead for investigation: compare latency, reconstruction error, and gradient behavior on real data on both sides of the threshold, then assess whether to specify a driver explicitly. Key points to follow are whether a change to the default policy is merged, which data types it covers, and how the new strategy performs on ill-conditioned matrices.