3D Bounding Box Estimation Using Deep Learning and Geometry

The problem of 3D object detection is of particular importance in robotic applications that require decision making or interactions with objects in the real world. While recently developed 2D detection algorithms are capable of handling large variations in viewpoint and clutter, accurate 3D object detection largely remains an open problem despite some promising recent work. They first regress relatively stable 3D object properties using a deep convolutional neural network and then combines these estimates with geometric constraints provided by a 2D object bounding box to produce a complete 3D bounding box. Given estimated orientation and dimensions and the constraint that the projection of the 3D bounding box fits tightly into the 2D detection window, they recover the translation and the object's 3D bounding box. In order to study this article mathematically, we need a coordinate system.