In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models of images by probing the learning algorithm with carefully designed training datasets. By measuring properties of the learned distribution, we are able to find interesting patterns of generalization. We verify that these patterns are consistent across datasets, common models and architectures.

To do so, we first form a family of unnormalized densities (or a "path") parameterized by 2 [0, 1] between the two distributions of interest

A question that arises is what structures should be represented from the data.

Dear Reviewers, we thank you for taking the time to provide valuable feedback. Below we address the main issues raised. Its performance depends on our ability to predict the distribution over future frames with low entropy. We will emphasize these aspects more in a revised version. RNNs to model dynamics in the latent space.

Neural Information Processing Systems http://nips.cc/

However, current methods for evaluating such models remain incomplete: standard likelihood-based metrics do not always apply and rarely correlate with perceptual fidelity, while sample-based metrics, such as FID, are insensitive to overfitting, i.e., inability to generalize beyond the training set.

Atpresent, however, there are few flexible and provably exchangeable deep generative models to solve this problem.

Neural Information Processing Systems http://nips.cc/