Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

In this paper, we provide deeper insights into Anderson acceleration in reinforcement learning by establishing its connection with quasi-Newton methods for policy iteration and improved convergence guarantees under the assumptions that the Bellman operator is differential and non-expansive.