Optimization
Stochastic Spectral and Conjugate Descent Methods
Dmitry Kovalev, Peter Richtarik, Eduard Gorbunov, Elnur Gasanov
An increasing array of learning and training tasks reduce to optimization problem in very large dimensions. The state-of-the-art algorithms in this regime are based on randomized coordinate descent (RCD) . V arious acceleration strategies were proposed for RCD in the literature in recent years, based on techniques such as Nesterov's momentum [