PerAct2: A Perceiver Actor Framework for Bimanual Manipulation Tasks

Humans seamlessly manipulate and interact with their environment using both hands. With both hands, humans achieve greater efficiency through enhanced reachability and can solve more sophisticated tasks. Despite the recent advances in grasping and manipulation planning [3, 4] the investigation of bimanual manipulation remains an under-explored area, especially in terms of learning a manipulation policy. Unlike tasks that require grasping or manipulation with a single hand, bimanual manipulation Figure 1: Selected bimanual tasks from the benchmark introduces a layer of complexity due to the as well as real-world examples. Due to the need for spatial and temporal coordination and architecture design the method can easily be transferred a deep understanding of the task at hand. This to other robots as the policy outputs a 6-D complexity is compounded by the dynamic nature pose and is agnostic to the underlying controller. of real-world tasks, where the state of the environment and the objects within it are constantly changing, demanding continuous adjustment and coordination between both arms.