The Collusion of Memory and Nonlinearity in Stochastic Approximation With Constant Stepsize

SA is not merely an overlook--as argued below, this setting is significantly more challenging. Our Contributions In this work, we study constant-stepsize SA with both Markovian data and nonlinear update. In Section 3, we elucidate the new challenges that arise from the simultaneous presence of these two structures, which break key steps in previous analyses of the i.i.d. or linear setting.