Specifically,becauseaQfunctionis defined with respect toaparticular policy,constructingPˆQ requires selection ofareference policy or distribution over policies.

The vertical separators correspond to loading network weights and replay buffer for fine-tuning while offline pre-training on replay buffer using QDagger (Section 4.1) for reincarnation. Shaded regions show 95% confidence intervals.

Planned Path Graph + SF Update Graph + 4. Use random policy to explore

A reinforcement learning (RL) agent is tasked with two fundamental, interdependent problems: exploration(howtodiscoverusefuldata),andcreditassignment(howtoincorporateit). The simplest way of estimating the value function is by averaging returns (futurediscountedsumsofrewards)startingfromtaking ainx.

Neural Information Processing Systems http://nips.cc/