The researchers taught the robot, called Mobile ALOHA (an acronym for "a low-cost open-source hardware teleoperation system for bimanual operation"), seven different tasks requiring a variety of mobility and dexterity skills, such as rinsing a pan or giving someone a high five. To teach the robot how to cook shrimp, for example, the researchers remotely operated it 20 times to get the shrimp into the plan, flip it, and then serve it. They did it slightly differently each time so the robot learned different ways to do the same task, says Zipeng Fu, a PhD Student at Stanford, who was project co-lead. The robot was then trained on these demonstrations, as well as other human-operated demonstrations for different types of tasks that have nothing to do with shrimp cooking, such as tearing off a paper towel or tape collected by an earlier ALOHA robot without wheels, says Chelsea Finn, an assistant professor at Stanford University, who was an advisor for the project. This "co-training" approach, in which new and old data are combined, helped Mobile ALOHA learn new jobs relatively quickly, compared with the usual approach of training AI systems on thousands if not millions of examples.

Imitation learning from human demonstrations has shown impressive performance in robotics. However, most results focus on table-top manipulation, lacking the mobility and dexterity necessary for generally useful tasks. In this work, we develop a system for imitating mobile manipulation tasks that are bimanual and require whole-body control. We first present Mobile ALOHA, a low-cost and whole-body teleoperation system for data collection. It augments the ALOHA system with a mobile base, and a whole-body teleoperation interface. Using data collected with Mobile ALOHA, we then perform supervised behavior cloning and find that co-training with existing static ALOHA datasets boosts performance on mobile manipulation tasks. With 50 demonstrations for each task, co-training can increase success rates by up to 90%, allowing Mobile ALOHA to autonomously complete complex mobile manipulation tasks such as sauteing and serving a piece of shrimp, opening a two-door wall cabinet to store heavy cooking pots, calling and entering an elevator, and lightly rinsing a used pan using a kitchen faucet. Project website: https://mobile-aloha.github.io