Under the Hood of the Variational Autoencoder (in Prose and Code)

In Part I of this series, we introduced the theory and intuition behind the VAE, an exciting development in machine learning for combined generative modeling and inference--"machines that imagine and reason." To recap: VAEs put a probabilistic spin on the basic autoencoder paradigm--treating their inputs, hidden representations, and reconstructed outputs as probabilistic random variables within a directed graphical model. With this Bayesian perspective, the encoder becomes a variational inference network, mapping observed inputs to (approximate) posterior distributions over latent space, and the decoder becomes a generative network, capable of mapping arbitrary latent coordinates back to distributions over the original data space. The beauty of this setup is that we can take a principled Bayesian approach toward building systems with a rich internal "mental model" of the observed world, all by training a single, cleverly-designed deep neural network. These benefits derive from an enriched understanding of data as merely the tip of the iceberg--the observed result of an underlying causative probabilistic process.