We show that the proposed algorithms converge to the (regularized) global optimal solution, andmoreover,theirratesofconvergence areofpolynomial orderinthe online setting and exponential order inthe finite sample setting, respectively.
Stochastic Proximal Gradient (SPG) methods have been widely used for solving optimization problems with a simple (possibly non-smooth) regularizer in machine learning and statistics.