Learning Attractor Landscapes for Learning Motor Primitives

Many control problems take place in continuous state-action spaces, e.g., as in manipulator robotics, where the control objective is often definedas finding a desired trajectory that reaches a particular goal state. While reinforcement learning offers a theoretical framework tolearn such control policies from scratch, its applicability to higher dimensional continuous state-action spaces remains rather limited to date. Instead of learning from scratch, in this paper we suggest to learn a desired complex control policy by transforming an existing simple canonical control policy. For this purpose, we represent canonical policies in terms of differential equations with well-defined attractor properties. By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated withoutlosing the stability properties of the canonical system.