Generative Adversarial Networks (GANs): What it can generate and What it cannot?

A generative model is trained to learn the underlying distribution of the data. The idea behind having such models is not to memorize the entire data, but to learn those specific semantic and structural properties which help the model create new samples. These samples need not belong to the training set, yet can convincingly become a part of it. The other popular models such as Restricted Boltzmann Machines (RBMs) [9], Variational Auto-encoders (VAEs) [11] make use of latent variables as a hidden representation of the data samples. These models specify an explicit parameterized log-likelihood functions representing the data. The parameters are learned from the data. Estimating the maximum likelihood of the parameters requires integrating over the entire space of latent variables, which is intractable. Hence approximation techniques are used which may not always yield the best results. On the other hand, Generative Adversarial Networks, GANs, are one of the few implicit probabilistic models which define a stochastic procedure that directly generates data from a latent variable that belongs to a lower dimensional space.