Augment-Connect-Explore: a Paradigm for Visual Action Planning with Data Scarcity

Abstract-- Visual action planning particularly excels in applications where the state of the system cannot be computed explicitly, such as manipulation of deformable objects, as it enables planning directly from raw images. Even though the field has been significantly accelerated by deep learning techniques, a crucial requirement for their success is the availability of a large amount of data. In this work, we propose the Augment-Connect-Explore (ACE) paradigm to enable visual action planning in cases of data scarcity. We build upon the Latent Space Roadmap (LSR) framework which performs planning with a graph built in a low dimensional latent space. In particular, ACE is used to i) Augment the available training dataset by autonomously creating new pairs of datapoints, ii) create new Figure 1: Overview of our ACE paradigm: (1) gaining new similar unobserved Connections among representations of states in the datapairs by Augmenting existing ones, (2) building unseen latent graph, and iii) Explore new regions of the latent space in a Connections in the latent space, and (3) efficiently Exploring new targeted manner.