Negative Sampling in Variational Autoencoders

We propose negative sampling as an approach to improve the notoriously bad out-of-distribution likelihood estimates of V ariational Autoencoder models. Our model pushes latent images of negative samples away from the prior. When the source of negative samples is an auxiliary dataset, such a model can vastly improve on baselines when evaluated on OOD detection tasks. Perhaps more surprisingly, we present a fully unsupervised variant that can also significantly improve detection performance: using the output of the generator as negative samples results in a fully unsupervised model that can be interpreted as adversarially trained. Learning semantically meaningful and useful representations for downstream tasks in an unsupervised manner is a big promise of generative modeling. While a plethora of work demonstrates the effectiveness of deep generative models in this regard, recent work of Nalisnick et al. (2019a) and Choi et al. (2018) show that these models often fail even at a task that is supposed to be close to their original goal of learning densities. V ariational Autoencoders, PixelCNN and flow-based models cannot distinguish common objects like cats and dogs from house numbers.