Deep Neural Networks Tend To Extrapolate Predictably

The prevailing belief in machine learning posits that deep neural networks behave erratically when presented with out-of-distribution (OOD) inputs, often yielding predictions that are not only incorrect, but incorrect with high confidence [19, 37]. However, there is some evidence which seemingly contradicts this conventional wisdom - for example, Hendrycks and Gimpel [24] show that the softmax probabilities outputted by neural network classifiers actually tend to be less confident on OOD inputs, making them surprisingly effective OOD detectors. In our work, we find that this softmax behavior may be reflective of a more general pattern in the way neural networks extrapolate: as inputs diverge further from the training distribution, a neural network's predictions often converge towards a fixed constant value. Moreover, this constant value often approximates the best prediction the network can produce without observing any inputs, which we refer to as the optimal constant solution (OCS). We call this the "reversion to the OCS" hypothesis: Neural networks predictions on high-dimensional OOD inputs tend to revert towards the optimal constant solution.