Deep learning with point clouds

If you've ever seen a self-driving car in the wild, you might wonder about that spinning cylinder on top of it. It's a "lidar sensor," and it's what allows the car to navigate the world. By sending out pulses of infrared light and measuring the time it takes for them to bounce off objects, the sensor creates a "point cloud" that builds a 3D snapshot of the car's surroundings. Making sense of raw point-cloud data is difficult, and before the age of machine learning it traditionally required highly trained engineers to tediously specify which qualities they wanted to capture by hand. But in a new series of papers out of MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), researchers show that they can use deep learning to automatically process point clouds for a wide range of 3D-imaging applications.