[D] How to get NNs to converge to the prior for unseen outliers? • r/MachineLearning

There's a tendency for classifier NNs to be overconfident about outliers -- generally they extrapolate to arbitrary values outside of the training data distribution. This is only exacerbated by the use of ReLUs and the like, which grow indefinitely and generally tend to P(x c)-- 1 for some category when x moves away from the training set. In my application, I would like the classifier to give a flat distribution for inputs outside of the training set; are there any tried and tested ways of doing that?