The Lottery Ticket Hypothesis is a conjecture that every large neural network contains a subnetwork that, when trained in isolation, achieves comparable performance to the large network.

We study the problem of inverting a deep generative model with ReLU activations. Inversion corresponds to finding a latent code vector that explains observed measurements as much as possible.

An important class of problems involves training deep neural networks with sparse prediction targets of very high dimension D . These occur naturally in e.g.

However, no direct empirical evidence for this explanation or even an approach for obtaining such evidence were given. In this work we design experiments to test the key ideas in this theory.

Asymptotically optimal channel codes have been developed by mathematicians for communicating under canonical models after over 60 years of research.

We thank the reviewers for their insightful comments. Please find the detailed responses below. TurboAE has a linear complexity in the block length, in runtime and computation. TurboAE and traditional turbo codes run on both GPU and CPU in the revision. We agree with the reviewer that the statement is confusing and misleading.

Neural Information Processing Systems http://nips.cc/