We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input $x$ and target $y$, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.

Humans imagine things. We live our lives in great measure by imagining circumstances a bit different from what we find, and then (again using imagination) we explore what it might take to bring those circumstances about, or what it might be like to live in such circumstances. We do this regarding issues large and small, all day long, every day. It is how we operate, and it gives us a huge leg up in detecting and repairing our own confusion as we negotiate this complex dynamic world. It is also quite different from how our artificial systems operate.