Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

In deep reinforcement learning, policy optimization methods need to deal with issues such asfunction approximation andthereuse ofoff-policydata.

Consider a player that in each of T rounds chooses one of K arms. An adversary chooses the cost of each arm in a bounded interval, and a sequence of feedback delays {dt} that are unknown to the player. After picking arm at at round t, the player receives the cost of playing this arm dt rounds later. In cases where t + dt > T, this feedback is simply missing.

In the most basic setting of this problem, an agent has to pull one among a finite set of arms (or actions), and she receives a reward that depends on the chosen action.

Building on this result, we show several examples of how variance of predictions can be exploited in learning.

We present a computationally efficient algorithm for learning in this framework that simultaneously achieves near-optimal regret bounds in both stochastic and adversarial environments. The bound against oblivious adversaries is O( αT), where T is the time horizon andα is the independence number of the feedback graph.

The standard assumption in reinforcement learning (RL) is that agents observe feedback for their actions immediately. However, in practice feedback is often observedindelay.