Reviews: Single-Model Uncertainties for Deep Learning

This work presents ways to obtain estimates of the aleatoric and epistemic uncertainties of deep neural networks. The aleatoric uncertainty is estimated by learning the quantiles of the target variable via Simultaneous Quantile Regression (SQR); it minimizes the pinball loss where the target quantile is randomly sampled in every training iteration. The epistemic uncertainty is implicitly estimated by Orthonormal Certificates (OCs); these are functions that are trained to map in-distribution examples to zero whereas out-of-distribution examples to non-zero values. The authors also provide tail bounds for the OCs in the case of Gaussian input data, which does provide some intuition about the behaviour. Simplicity is a benefit of these estimators and the authors demonstrate their performance on regression and classification tasks.