A General Pipeline for 3D Detection of Vehicles

Abstract-- Autonomous driving requires 3D perception of vehicles and other objects in the in environment. Much of the current methods support 2D vehicle detection. This paper proposes a flexible pipeline to adopt any 2D detection network and fuse it with a 3D point cloud to generate 3D information with minimum changes of the 2D detection networks. To identify the 3D box, an effective model fitting algorithm is developed based on generalised car models and score maps. A two-stage convolutional neural network (CNN) is proposed to refine the detected 3D box. This pipeline is tested on the KITTI dataset using two different 2D detection networks. The 3D detection results based on these two networks are similar, demonstrating the flexibility of the proposed pipeline. The results rank second among the 3D detection algorithms, indicating its competencies in 3D detection. I. INTRODUCTION Vision-based car detection has been well developed and widely implemented using deep learning technologies. The KITTI [1] benchmark site reports that the state of the art algorithms are able to achieve 90% average precision (AP). However, for autonomous vehicles, car detection in 2D images is not sufficient to provide enough information for the vehicle to perform planning and decision making due to the lack of depth data.