DittoGym: Learning to Control Soft Shape-Shifting Robots

Robot co-design, where the morphology of a robot is optimized jointly with a learned policy to solve a specific task, is an emerging area of research. It holds particular promise for soft robots, which are amenable to novel manufacturing techniques that can realize learned morphologies and actuators. Inspired by nature and recent novel robot designs, we propose to go a step further and explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We unify morphology change, locomotion, and environment interaction in the same action space, and introduce an appropriate, coarse-to-fine curriculum that enables us to discover policies that accomplish fine-grained control of the resulting robots. We also introduce Ditto-Gym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes to accomplish the tasks. Finally, we evaluate our proposed coarse-to-fine algorithm on DittoGym and demonstrate robots that learn to change their morphology several times within a sequence, uniquely enabled by our RL algorithm. More results are available at https://dittogym.github.io. Over millions of years, morphologies of species change as a function of evolutionary pressures (Minelli, 2003; Raff, 2012). In robotics, this process of evolution has inspired the task of robot co-design: the joint optimization of a robot's morphology and a control policy that best enable the robot to accomplish a given task (Gupta et al., 2022; Wang et al.; Ha, 2019; Yuan et al., 2021). Yet, in nature, creatures do not only change their morphology over millions of years as a function of evolution. Almost all living beings go through a process of morphology changes even in their lifetime. These changes can be dramatic in magnitude, like when a mighty tree grows from a tiny sapling, but they can also be dramatic in form, like across the many examples of metamorphosis, where frogs, for instance, go through a water-dwelling stage with a tail for propulsion, to then lose their tail and grow legs to live on land (Rose, 2005; Hofmann et al., 2003).