Offline Learning of Controllable Diverse Behaviors

Accepted to the Generative Models for Robot Learning Workshop at ICLR 2025O FFLINEL EARNING OF C ONTROLLABLED IVERSE B E-HAVIORS Mathieu Petitbois *,1, R emy Portelas 1, Sylvain Lamprier 2, Ludovic Denoyer 3 1 Ubisoft La Forge 2 University of Angers 3 H Company A BSTRACT Imitation Learning (IL) techniques aim to replicate human behaviors in specific tasks. While IL has gained prominence due to its effectiveness and efficiency, traditional methods often focus on datasets collected from experts to produce a single efficient policy. Recently, extensions have been proposed to handle datasets of diverse behaviors by mainly focusing on learning transition-level diverse policies or on performing entropy maximization at the trajectory level. While these methods may lead to diverse behaviors, they may not be sufficient to reproduce the actual diversity of demonstrations or to allow controlled trajectory generation. To overcome these drawbacks, we propose a different method based on two key features: a) Temporal Consistency that ensures consistent behaviors across entire episodes and not just at the transition level as well as b) Controllability obtained by constructing a latent space of behaviors that allows users to selectively activate specific behaviors based on their requirements. We compare our approach to state-of-the-art methods over a diverse set of tasks and environments. For robotics, learning from human experts allows to reach human-level performance without any controller hard coding or expensive interaction with simulated or real environments.