Once it has used deep learning to identify an object, the robot uses a knowledge base to pull out more detailed information that helps it carry out its orders. For example,when it identifies an object as a car, it consults a list of facts relating to cars: a car is a vehicle, it has wheels and an engine, and so on. These facts need to be hand-coded and are time consuming to compile, however, and Stump says the team is looking into ways to streamline this.

Despite what you might see in movies, today's robots are still very limited in what they can do. They can be great for many repetitive tasks, but their inability to understand the nuances of human language makes them mostly useless for more complicated requests. For example, if you put a specific tool in a toolbox and ask a robot to "pick it up," it would be completely lost. Picking it up means being able to see and identify objects, understand commands, recognize that the "it" in question is the tool you put down, go back in time to remember the moment when you put down the tool, and distinguish the tool you put down from other ones of similar shapes and sizes. Recently researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) have gotten closer to making this type of request easier: In a new paper, they present an Alexa-like system that allows robots to understand a wide range of commands that require contextual knowledge about objects and their environments.