Reviews: Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks

This paper proposes a simple and efficient block coordinate descent (BCD) algorithm with a novel Tikhonov regularization for training both dense and sparse DNNs with ReLU. They show that the proposed BCD algorithm converges globally to a stationary point with an R-linear convergence rate of order one and performs better than all the SGD variants in experiments. However, the motivations of using Tikhonov regularization and block coordinate descent are not very clear. The technical parts are hard to follow due to the absence of many details. The presented results are far from state-of-the-art. In this sense, I am not sure whether the proposed method can be applied to real "DNNs".