On weight and variance uncertainty in neural networks for regression tasks

Bayesian Neural Networks (BNNs) have been introduced and comprehensively discussed by many authors (among others see [14, 2]). BNNs are suitable for modeling uncertainty by considering values of the parameters that might not be learned by the available data. This is achieved by randomizing unknown parameters, such as weights and biases while incorporating prior knowledge. Such an approach naturally regularizes the model and helps prevent overfitting, a common challenge in neural networks (NNs). The BNNs can also be considered as a suitable and more reliable alternative to ensemble learning methods [15], including bagging and boosting of the neural networks.