Helping robots learn from each other

Just like a Transformer-based language model predicts the next word based on trends and patterns it sees in text, RT-1 has been trained on robotic perception data – images – and corresponding actions so it can identify the next most likely behavior a robot should take. For example, the model is trained on many examples of how a robot should pick up a banana, the robot would learn how to identify and pick up a banana even when the banana is seen in a new kitchen surrounded by objects it's never seen before. This approach enables the robot to generalize what it's learned to new tasks, handling new objects and environments based on experiences in its training data -- a rare feat for robots, which are typically strictly coded for narrow tasks.