Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization

Stochastic optimization is an essential component of most state-of-the-art the machine learning techniques. Sources of stochasticity in machine learning optimization include handling large datasets, approximating expectations, and modelling uncertain dynamic environments. The seminal work of (Robbins & Monro, 1985) showed that, under certain conditions, it is possible to use gradient-based optimization in the presence of randomness. However, it is well-known that gradient randomness has an adverse effect on the performance of stochastic gradient descent (SGD) (Wang, Chen, Smola, & Xing, 2013). As a result, the construction of methods to reduce gradient noise is an active field of research (Wang et al., 2013; Mandt & Blei, 2014; Grathwohl, Choi, Wu, Roeder, & Duvenaud, 2017; Roeder, Wu, & Duvenaud, 2017).