Importance weighted generative networks

Deep generative models have important application in many fields: we can automatically generate illustrations for text [32]; simulate video streams [30] or molecular fingerprints [17]; and create privacy-preserving versions of medical time-series data [9]. Such models use a neural network to parametrize a function G(Z), which maps random noise Z to a target probability distribution P. This is achieved by minimizing a loss function between simulations and data, which is equivalent to learning a distribution over simulations that is indistinguishable from P under an appropriate two-sample test. In this paper we focus on Generative Adversarial Networks (GANs) [11, 2, 3, 19], which incorporate an adversarially learned neural network in the loss function; however the results are also applicable to non-adversarial networks [8, 20]. An interesting challenge arises when we do not have direct access to i.i.d.