ETH Zürich Identifies Priors That Boost Bayesian Deep Learning Models

It's well known across the machine learning community that choosing the right prior -- an initial belief re an event expressed in terms of a probability distribution -- is crucial for Bayesian inference. Many recent Bayesian deep learning models however resort to established but uninformative or weak informative priors that may have detrimental consequences on their models' inference abilities. In the paper Priors in Bayesian Deep Learning: A Review, a research team from ETH Zürich presents an overview of different priors for (deep) Gaussian processes, variational autoencoders, and Bayesian neural networks. The team proposes that well-chosen priors can actually achieve theoretical and empirical properties such as uncertainty estimation, model selection and optimal decision support; and provides guidance on how to choose them. The main idea of Bayesian models is to infer a posterior distribution over the parameters of a model based on a prior probability for some observed data.