UMAD: Unsupervised Mask-Level Anomaly Detection for Autonomous Driving

Although great achievements have been made in autonomous driving, reacting to the unknown remains a significant challenge [1, 2]. Heidecker et al. [1] categorize anomalies into the sensor, content, and temporal layer: Anomalies in the sensor layer are related to sensory abnormalities, anomalies in the content layer regard abnormalities in single observations, such as atypical objects, and the temporal layer considers behavioral anomalies in the context of multiple frames. Classically, anomaly detection is based on highly specialized methods, focusing on the content layer [3, 4, 5]. However, a perpendicular line of work tries to learn a more general understanding of the world. Generative world models have shown promising results in autonomous driving [6, 7, 8, 9, 10]. They embed sensory data into latent states, reconstruct observations based on those, and predict action-conditioned future states. For anomaly detection, however, they have not been utilized yet [11].