The Phenomenon of Policy Churn

Neural Information Processing Systems 

We characterise the phenomenon empirically, verifying that it is not limited to specific algorithm or environment properties.