One of the challenges in processing sequential data is accurately capturing long-range input dependencies -- interactions between symbols that are far apart in the sequence.

Recently, we have witnessed great progress in the field of medical imaging classification by adopting deep neural networks. However, the recent advanced models still require accessing sufficiently large and representative datasets for training, which is often unfeasible in clinically realistic environments.

We consider the task of building compact deep learning pipelines suitable for deployment on storage and power constrained mobile devices. We propose a unified framework to learn a broad family of structured parameter matrices that are characterized by the notion of low displacement rank. Our structured transforms admit fast function and gradient evaluation, and span a rich range of parameter sharing configurations whose statistical modeling capacity can be explicitly tuned along a continuum from structured to unstructured. Experimental results show that these transforms can significantly accelerate inference and forward/backward passes during training, and offer superior accuracy-compactness-speed tradeoffs in comparison to a number of existing techniques. In keyword spotting applications in mobile speech recognition, our methods are much more effective than standard linear low-rank bottleneck layers and nearly retain the performance of state of the art models, while providing more than 3.5-fold compression.

In this work, we focus on an even more fundamental question on GANs and other generative models which is not yet fully addressed.

It utilizes a new unbiased error estimate that is based on adversarial examples generated from the test data and importance weighting.

Neural Information Processing Systems http://nips.cc/

Work done at Princeton as a Fulbright Scholar.

A recent line of research has highlighted the existence of a "double descent"