GitHub Repo
PyTorch Community Reports Zero Gradients in Inverse Hyperbolic Functions, Triggered by float16 Input of 256
A PyTorch community report on October 24 says `asinh` and `acosh` may return zero gradients for certain inputs because of intermediate overflow. The report includes a reproduction and a proposed fix, but the analysis was AI-assisted, and an official fix and the scope of impact remain unconfirmed.
A numerical stability report appeared in the PyTorch repository on October 24: in CPU eager mode on version 2.14.1, torch.asinh and torch.acosh can produce zero gradients during backpropagation for certain finite inputs, even though the forward results are normal. At the time of review, the issue remained open, with no official fix release. Community report
The key case in the report uses a float16 input of 256: both operations produce zero gradients, while temporarily promoting the operation to float32 yields approximately 0.00390625. The author attributes this to the derivative formulas first squaring the input and then taking the reciprocal of the square root; squaring 256 exceeds the float16 range, so the intermediate overflow erases the result even though the final gradient is representable. Reproduction and analysis
This error can propagate through a computation graph. The report demonstrates asinh(256 * u) with u=1, where the gradient with respect to u, which should be approximately one, also becomes zero. For engineers who use half-precision tensors directly and include these functions in differentiable transformations, this means checking only whether forward outputs are finite may miss a break in the training signal. Composed operation example
The official numerical accuracy documentation already explains that inputs and final answers being representable in a data type does not mean intermediate steps cannot overflow; different backends may also use different algorithms. The report is therefore best treated as a stability issue that could be improved; it does not establish that all half-precision training is affected. Official documentation
The author proposes a candidate fix that handles low-precision intermediate values at higher precision, but the report explicitly states that Codex assisted with preparing the analysis, tests, and fix. CUDA and standard AMP behavior have not yet been verified. Engineering teams can first compare gradients with and without a local float32 operation, then track maintainer confirmation, fix integration, and cross-device testing before deciding whether to adjust deployments. Verification limitations