Reviews: Learning from Trajectories via Subgoal Discovery

The paper describes a framework to learn from different expert trajectories how to decompose a task into smaller sub-goals. This is used to define a reward function and imitation learning is first used to learn a policy followed by reinforcement learning given the sub-goals. Also a module to deal with new states based on one-class learning is used to provide robustness to the system. A nice feature of the system is that it can learn tasks even with sub-optimal trajectories. One limitation is that the system assumes a sequential order of the sub-goals and all the trajectories must start from the same initial state distribution. This means that the system is unable to deal with sequences following different paths or from different initial states.