Hybrid-Diffusion Models: Combining Open-loop Routines with Visuomotor Diffusion Policies

Abstract-- Despite the fact that visuomotor-based policies obtained via imitation learning demonstrate good performances in complex manipulation tasks, they usually struggle to achieve the same accuracy and speed as traditional control based methods. In this work, we introduce Hybrid-Diffusion models that combine open-loop routines with visuomotor diffusion policies. We develop T eleoperation Augmentation Primitives (T APs) that allow the operator to perform predefined routines, such as locking specific axes, moving to perching waypoints, or triggering task-specific routines seamlessly during demonstrations. Our Hybrid-Diffusion method learns to trigger such T APs during inference. All experimental videos are available on the project's website: https://hybriddiffusion. github.io/ Advances in Imitation Learning [1]-[4] have propelled autonomous manipulation capabilities to tackling complex tasks such as spreading sauce on a pizza [1], opening a capped bottle [5], inserting a hanger into a T -shirt [3], and mounting a gear on a bike [4].