Reviews: Variational Bayes on Monte Carlo Steroids

This paper provides theoretical bounds that are tighter than existing variational bounds for the problem of learning latent variable models. The authors extend applied existing theory of hash-based learning and amortized inference to design a black-box learning algorithm. They later applied it to learning a Sigmoid Belief Network. The main advantage to this approach seems to be the partitioning of the search space for posterior distributions into buckets/subsets that are faster to search than with a typical sampling method. The proposed inference scheme then leverages mean-field inference (used heavily in the context of variational inference) within each subset. One of the main technical contributions is the tighter bound on the likelihood using two aggregate estimators which was an extension of an existing work (specific to undirected graphical models) to the directed models setting.