Reviews: Deep Neural Nets with Interpolating Function as Output Activation

This paper develops a new data-dependent output activation function base on interpolation function. It is a nonparametric model based on a subset of training data. The activation function is defined in an implicit manner by solving a set of linear equations. Therefore, it cannot be solved directly by backpropagation. Instead it proposes an auxiliary network with linear output to approximate the gradient.