Review for NeurIPS paper: Proximal Mapping for Deep Regularization

Summary and Contributions: This work proposes the use of proximal mapping to introduce certain data-dependent regularizers on neural network activations. The authors introduce two different regularization methods based on this idea. The first is a regularization on outputs of a recurrent network (LSTM) to encourage robustness to perturbations in the input. This regularizer has a closed form solution, though second order derivatives are required. The second regularization method introduced controls correlation between activations of hidden layers on two different data sets, similar to deep CCA (DCCA).