Reviews: Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee

The paper presents a technique to sparsify a deep ReLU neural network by solving a sequence of convex problems at each layer. The convex problem finds the sparsest set of weights that approximates the mapping from one layer to another. The ReLU nonlinearity is dealt with by treating the activated and deactivated cases as two separate sets of constraints in the optimization problem, thus, bringing convexity. Two variants are provided, one that considers each layer separately, and another that carries the approximation in the next layer optimization problem to give the chance to the next layer to counterbalance this error. In both cases, the authors provide bounds on the approximation error after sparsification.