Reinforcement Learning in Time-Varying Systems: an Empirical Study