Enhanced Variational Inference with Dyadic Transformation

A generative model is an unsupervised learning approach that is able to learn a domain by processing a large amount of data from it and then generate new data like it (Hinton and Ghahramani, 1997;Yu et al., 2018). VAE, together with Generative Adversarial Networks (Goodfellow et al., 2016) and Deep Autoregressive Networks(Gregor et al., 2013), are amongst the most powerful and popular generative model techniques. VAE has been successfully applied in many domains, such as image processing (Pu et al., 2016), natural language processing (Semeniuta etal., 2017), and cybersecurity (Chandy et al., 2019). A VAE works by maximizing a variational lower bound of the likelihood of the data (Kingma and Welling, 2013). A VAE has two halves: a recognition model (an encoder) and a generative model(a decoder). The recognition model learns a latent representation of the input data, and the generative model learns to transform this representation back into the original data. The recognition and generative models are jointly trained by optimizing theprobability of the input data using stochastic gradient ascent. Application of the VAE involves selection of an approximate posterior distribution for the latent variables.