PyTorch Community Reports CPU Variance Overflow; Large-Value Constant Tensors May Return Infinity

On October 8, the PyTorch community reported a numerical stability issue: on CPU, calling torch.var or torch.std without specifying a dimension may return inf for some large-value constant tensors. The reporter listed two 2.15 development builds, as well as aarch64 macOS and x86_64 Linux. At the time of review, the issue remained open and listed no associated fix. Community report

The minimal example requires only a float64 tensor containing six values, each equal to 1e200. The report shows that torch.var(x) returns infinity, while torch.var_mean(x)[0] returns zero for the same input. The official documentation defines both with the same variance formula and a default correction of correction=1. Six identical samples should have zero variance, so the difference cannot be explained by the degrees-of-freedom setting. var documentation, var_mean documentation

According to the reporter’s analysis of the code paths, the difference comes from the underlying reduction algorithms. The CPU fast path for whole-tensor var first computes the mean, then accumulates squared deviations. Rounding error in the mean leaves a huge residual, which exceeds the representable range of float64 when squared. var_mean instead follows a Welford update path, which avoids this overflow in the reported case. This root-cause analysis is currently the reporter’s interpretation. Technical analysis and reproduction

For engineering teams, this offers a concrete lead when investigating statistical anomalies: if data preprocessing or research code produces an infinite variance, compare the two APIs and hold input size, precision, and execution environment constant. This diagnostic suggestion is inferred from the case. There is no evidence that general model training is broadly affected, and substituting one API for the other should not be treated as a universal fix.

Next, watch for whether maintainers confirm the scope across released versions and whether a fix addresses extreme magnitudes while preserving the fast path’s performance. Validation should also cover different tensor sizes and reductions along specified dimensions, so that a single case returning to normal is not taken as proof that the entire class of numerical issues has been resolved.