During initial iterations of training in most Reinforcement Learning (RL) algorithms, agents perform asignificant number ofrandom exploratory steps.

In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired.

Toour knowledge, such guarantees in this setting are providedforthefirsttime.

Toour knowledge, such guarantees in this setting are providedforthefirsttime.

In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing thefullpicture.

P2SROisabletoparallelize PSROwith convergence guarantees bymaintaining ahierarchical pipeline ofreinforcement learning workers, each training against the policies generated by lower levels in the hierarchy.

We study the reinforcement learning problem for discounted Markov Decision Processes(MDPs)underthetabularsetting.