Camera-Lidar Projection: Navigating between 2D and 3D

Lidars and cameras are two essential sensors for perception and scene understanding. They build an environment in tandem and provide a means for detection and localisation of other objects, giving robots rich semantic information required for safe navigation. Many researchers have started exploring multi-modal deep learning model for precise 3D object detection. An interesting example would be an algorithm developed by Aptiv, PointPainting[1] Camera outperforms LIDAR when it comes to capturing denser and richer representation. From fig 2, looking at the sparse point cloud alone, it is relatively difficult to correctly identify the black box as a pedestrian.