Improve Uncertainty Estimation for Unknown Classes in Bayesian Neural Networks with Semi-Supervised /One Set Classification

One principle way to measure the uncertainty or confidence of the prediction is based on the statistic such as predictive mean, entropy and variance, using Bayesian machine learning, i.e. BNN if our model is a DNN[Nea12]. Beside capturing the parameters uncertainty, other methods works with modifying the loss function [DT18], [KG17], by which the model attempts to learn the heteroscedastic aleatoric uncertainty, i.e. uncertainty that depends on the input data (e.g. if there is an occlusion on the image object, our model will less likely to produce accurate prediction so we train our model such that it recognize "occlusion"). Experiments in previous paper [RBB18], [LPB17], the estimated predictive posterior is used to detect out-of-distribution data, or data from unknown class by measuring the its statistics. In other words, they expect data from unknown class to have high entropy and variance by using BNN.