Robot see, robot do: System learns after watching how-tos

Kushal Kedia (left) and Prithwish Dan (right) are members of the development team behind RHyME, a system that allows robots to learn tasks by watching a single how-to video. Cornell researchers have developed a new robotic framework powered by artificial intelligence – called RHyME (Retrieval for Hybrid Imitation under Mismatched Execution) – that allows robots to learn tasks by watching a single how-to video. RHyME could fast-track the development and deployment of robotic systems by significantly reducing the time, energy and money needed to train them, the researchers said. "One of the annoying things about working with robots is collecting so much data on the robot doing different tasks," said Kushal Kedia, a doctoral student in the field of computer science and lead author of a corresponding paper on RHyME. "That's not how humans do tasks. We look at other people as inspiration."