Reviews: Perturbative Black Box Variational Inference

Summary: The authors present a new variational objective for approximate Bayesian inference. The variational objective is nicely framed as an interpolation between classic importance sampling and the traditional ELBO-based variational inference. Properties of the variance of importance sampling estimator and ELBO estimators are studied and leveraged to create a better marginal likelihood bound with tractable variance properties. The new bound is based on a low-degree polynomial of the log-importance weight (termed the interaction energy). The traditional ELBO estimator is expressed as a first order polynomial in their more general framework.