Bayesian posterior approximation with stochastic ensembles

To further reduce the computational effort in evaluating the approximate posterior, stochastic methods such as We introduce ensembles of stochastic neural networks to Monte Carlo dropout [47] and DropConnect [51] inference approximate the Bayesian posterior, combining stochastic have also been used extensively [13, 14, 40]. They benefit methods such as dropout with deep ensembles. The stochastic from computationally cheaper inference by virtue of sampling ensembles are formulated as families of distributions and stochastically from a single model. Formulated as a trained to approximate the Bayesian posterior with variational variational approximation to the posterior, dropout samples inference. We implement stochastic ensembles based from a family of parameter distributions where parameters on Monte Carlo dropout, DropConnect and a novel nonparametric can be randomly set to zero. Although this particular family version of dropout and evaluate them on a toy of distributions might seem unnatural [11], it turns out problem and CIFAR image classification. For both tasks, that the stochastic property can help to find more robust regions we test the quality of the posteriors directly against Hamiltonian of the parameter space, a fact well-known from a long Monte Carlo simulations. Our results show that history of using dropout as a regularization method.