Feudal Reinforcement Learning

One way to speed up reinforcement learning is to enable learning to happen simultaneously at multiple resolutions in space and time. This paper shows how to create a Q-Iearning managerial hierarchy how to set tasks to their submanagersin which high level managers learn how to satisfy them. Sub-managerswho, in turn, learn understand their managers' commands. Theyneed not initially simply learn to maximise their reinforcement in the context of the current command. We illustrate the system using a simple maze task .. As the system learns how to get around, satisfying commands at the multiple than standard, flat, Q-Iearninglevels, it explores more efficiently and builds a more comprehensive map. 1 INTRODUCTION Straightforward reinforcement learning has been quite successful at some relatively thecomplex tasks like playing backgammon (Tesauro, 1992).