Note on the equivalence of hierarchical variational models and auxiliary deep generative models

In machine learning, there is an ongoing revival of the use of variational Bayes (VB) to deal with complex probabilistic models with hidden variables. The revival is driven by the use of stochastic methods to approximate the VB lower bound and associated gradients. See for example [1, 2, 5]. The advantages include automated inference [3] and also that they are applicable to a much wider class of probabilistic models. While the basic recipes are limited in the flexibility of the approximate hidden-variable posteriors, there are ongoing efforts to make them more flexible.