Variational Bayesian Decision-making for Continuous Utilities

A considerable proportion of research on Bayesian machine learning concerns itself with the fundamental task of inference, developing techniques for an efficient and accurate approximation of the posterior distribution p(θ D) of the model parameters θ conditional on observed data D. However, in most cases, this is not the end goal in itself. Instead, we eventually want to solve a decision problem of some kind and merely use the posterior distribution as a summary of the information provided by the data and the modeling assumptions. For example, we may want to decide to automatically shut down a process to avoid costs associated with its potential failure, yet might not necessarily care about whether we model all aspects of the process accurately. The focus on inference is justified by Bayesian decision theory Berger (1985). It formalizes the notion that the posterior distribution is sufficient for making optimal decisions under a utility. This is achieved by selecting decisions that maximize the expected utility, computed by integrating over the posterior.