Given the impressive results of deepreinforcement learning, weargueforaneedtomoreclearly understand the challenges inthis setting.

One of the most effectivecontinuous deep reinforcement learning algorithms is normalized advantage functions (NAF).

One of the phenomena of interest is that Q-learning (Watkins, 1989) is known to suffer from overestimation issues, since it takes a maximum operator overaset ofestimated action-values.

Most learning algorithms are not invariant to the scale of the signal that is being approximated. We propose to adaptively normalize the targets used in the learning updates. This is important in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior.

Neural Information Processing Systems http://nips.cc/

In particular, we look at commonalities and differences between parametric models and experience replay.

Credit assignment, i.e. determining how to correctly associate delayed rewards with states or state-action pairs, is a crucial problem for reinforcement learning (RL) agents ( Sutton and Barto, 2018).