Reviews: Piecewise Strong Convexity of Neural Networks

This paper shows that the quadratic loss with weight decay of deep ReLU networks is piecewise strongly convex on a nonempty open set where every critical point is a local minimum, and every local minimum is isolated. Initially the paper received mixed reviews, with two positive and one negative review. On the positive side, the contribution is found to be quite significant because it analyzes realistic networks (deep and non-linear). On the other hand, one reviewer had issues with the proof, and another with the experiments. The rebuttal addressed the issues raised by the reviewers, and the negative review updated the score.