Multimodal Detection of Unknown Objects on Roads for Autonomous Driving

Abstract--Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect (a) Lidar point cloud (b) Camera image unknown objects. In December 2021, Mercedes-Benz became the first automotive company to meet the legal requirements for a SAE the advantages of combining sensor modalities. However, a driver is still present fusion models are popular for classic object detection, they and must be ready to take over control. This is not the case often lack the awareness necessary for anomaly detection.