Optimal Options for Multi-Task Reinforcement Learning Under Time Constraints

However, even to learn to solve simple tasks it can require millions of interactions. A promising approach to improve the learning speed relies on the options framework [6] An option is a'chunk of behaviour' that is formally defined as an initiation set, establishing in which states the option is available; a policy, indicating which actions to perform in each state; and a termination condition, establishing when the option execution is terminated. RL systems can benefit from the use of options to support faster exploration and learning especially when rewards are sparse or when the solution to a problem involves recurring behaviours. An important open problem is how can an agent autonomously learn options that are useful to solve tasks drawn from a given task distribution. Recent approaches have searched options for specific optimisation problems but they have not studied how optimal options are affected by different task features such as limited learning time budgets, task rewards, initial states, and the learning algorithm used.