Towards closing the gap between the theory and practice of SVRG

Amongst the very first variance reduced stochastic methods for solving the empirical risk minimization problem was the SVRG method [13].