Stochastic Spectral and Conjugate Descent Methods

An increasing array of learning and training tasks reduce to optimization problem in very large dimensions. The state-of-the-art algorithms in this regime are based on randomized coordinate descent (RCD) . V arious acceleration strategies were proposed for RCD in the literature in recent years, based on techniques such as Nesterov's momentum [