Learning finite-dimensional coding schemes with nonlinear reconstruction maps

The problem of lossy compression is about constructing succinct representations of high-dimensional random vectors that retain the features of the data that are relevant for some subsequent task, such as reconstruction subject to a fidelity criterion or statistical inference. When the compressed representation is digital, with constraints imposed by the limitations on the speed of digital transmission oron the available storage space, the corresponding problem of lossy compression falls within the purview of rate-distortion theory[6] and the theory of vector quantization[15]. On the other hand, given recent advances in machine learning using deep neural nets[17], it is of interest to consider'analog' schemes for lossy compression that map the original high-dimensional data to a continuous representation of lower dimensionality[5], and where the reconstruction operations that send the compressed representation back to the original high-dimensional space are implemented bynonlinear maps with a given structure.