Giving robots human-like perception of their physical environments

To carry out such high-level tasks, researchers believe robots will have to be able to perceive their physical environment as humans do. "In order to make any decision in the world, you need to have a mental model of the environment around you," says Luca Carlone, assistant professor of aeronautics and astronautics at MIT. "This is something so effortless for humans. But for robots it's a painfully hard problem, where it's about transforming pixel values that they see through a camera, into an understanding of the world." Now Carlone and his students have developed a representation of spatial perception for robots that is modeled after the way humans perceive and navigate the world. The new model, which they call 3D Dynamic Scene Graphs, enables a robot to quickly generate a 3D map of its surroundings that also includes objects and their semantic labels (a chair versus a table, for instance), as well as people, rooms, walls, and other structures that the robot is likely seeing in its environment.