A Tutorial on Parametric Variational Inference

In Bayesian machine learning and statistics, the central object of interest is the posterior distribution found by Bayesian inference--combining prior beliefs with observations according to Bayes' rule. In simple cases, such as in conjugate models, this can be done exactly. But, general (nonconjugate) models require approximate inference techniques such as Monte Carlo or variational inference. These have complementary strengths and weaknesses, hence the most appropriate choice is application dependent. We focus on variational inference, which is on the one hand not guaranteed to be asymptotically exact but is on the other hand computationally efficient and scalable to high-dimensional models and large datasets.