Toour knowledge, such guarantees in this setting are providedforthefirsttime.

In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing thefullpicture.

P2SROisabletoparallelize PSROwith convergence guarantees bymaintaining ahierarchical pipeline ofreinforcement learning workers, each training against the policies generated by lower levels in the hierarchy.

We study the reinforcement learning problem for discounted Markov Decision Processes(MDPs)underthetabularsetting.

A major challenge in reinforcement learning is to determine which state-action pairs are responsible for future rewards that are delayed.