Variance-Reduced Stochastic Gradient Descent on Streaming Data

Ellango Jothimurugesan, Ashraf Tahmasbi, Phillip Gibbons, Srikanta Tirthapura

Neural Information Processing Systems 

Such a model is never "complete" but instead needs to be continuously updated as newer training data points arrive.