GitHub Repo
PyTorch Community Reports CPU Logit Precision Issue That Can Halve Results Near 0.5 with Half-Precision Inputs
A PyTorch 2.14 CPU community reproduction on October 6 found that rounding in intermediate logit calculations may cause about 50% relative error for certain half-precision inputs. The report includes an exhaustive enumeration of inputs and proposed patch code, but the patch has not been validated in an actual build.
On October 6, the PyTorch community reported a CPU numerical precision issue: torch.special.logit may lose important precision before taking the logarithm when processing float16 and bfloat16. The reported environment was PyTorch 2.14.0+cpu on an Intel Xeon with AVX512 and MKL. For a float16 input of about 0.499756, the output was about −0.000488, while the float64 reference was about −0.000977—a relative error close to 50%. Community reproduction
According to the official definition, logit computes log(x / (1 - x)), and eps can be used to clamp the input to a specified range. When x is close to 0.5, the ratio is close to 1 and the result of the logarithm is close to zero. Rounding in an intermediate value can therefore produce a substantial relative error even when the absolute difference is small. Setting a clamp range mainly addresses boundary values; it does not directly resolve precision issues in calculations within the range. API documentation
The reporter compared every representable input in the interval for both low-precision types against float64, and found that CPU subtraction and division round at the input dtype, while the CUDA code first promotes values to float. In the reporter’s test, converting with x.float(), computing logit, and then converting back to the original dtype reduced the maximum error to 0.5 ULP. ULP means the floating-point spacing around a given value. Tests and kernel analysis
For engineers using low-precision probabilities in calibration, statistical transformations, or CPU post-processing, this offers a concrete troubleshooting lead: when comparing results across devices, check both absolute and relative error, and use higher precision to establish a reference value. This is an engineering implication inferred from the computation mechanism; there is currently no evidence of an impact on full-model accuracy or training loss. PyTorch also notes that CPU and GPU results are not guaranteed to be bitwise identical, even with the same inputs. Numerical accuracy notes
The proposed patch in the report uses float for intermediate calculations and converts back to low precision only at the end. The author explicitly said they could not build PyTorch, and validated the patch arithmetic only with a NumPy simulation; they also did not measure performance. As of this check, the issue remains open. Follow-up should track upstream testing, review, and inclusion in a release; the attached code should not be treated as a shipped fix. Patch limitations