Variational Neural Networks

Abstract--Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other singlebin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods. BNNs consider a distribution do so, one needs the neural network to accompany its output P (w) over weights and sample different weights during each with a measurement of its corresponding uncertainty for each inference. VNNs consider a constant set of weights and use input it processes.