Igeood: An Information Geometry Approach to Out-of-Distribution Detection

Reliable out-of-distribution (OOD) detection is fundamental to implementing safer modern machine learning (ML) systems. By building on the geodesic (Fisher-Rao) distance between the underlying data distributions, our discriminator can combine confidence scores from the logits outputs and the learned features of a deep neural network. Deep neural networks (DNNs) reach the state-of-the-art in several classification tasks as they are known to generalize well on data with a distribution close to the training set. Whereas, in many practical applications, the training set does not reflect well enough the real-life environment (Quionero-Candela et al., 2009) which is often non-stationary and sometimes with unpredictable events. Therefore, matching the training scenario to reality can be impossible or too complex. The inability of machine learning (ML) models to adapt to non-stationary distributions could limit their adoption in mission-critical systems (e.g., autonomous devices, healthcare applications). Out-of-Distribution (OOD) or novelty detection is one of the main objectives in conceiving reliable ML systems (Amodei et al., 2016). A typical application is monitoring ML-based online services for periodically shifting distributions. However, tracking changes in the underlying data distribution is challenging as they contain unusual (irregular or unexpected) events and have large dimensions. For instance, relying on the intrinsic properties of ML models and their statistical behavior in the presence of in-distribution data is essential to identify OOD samples. Classic approaches to OOD detection consist of deriving metrics for detecting those abnormalities from the lens of ML models (e.g., softmax output, latent representations across layers), provided that often only a single test example is available. Furthermore, these metrics are subject to potential limitations inherent in practical scenarios depending on the level of access to information in the ML model, e.g., having access only to the last layer or to all intermediate layers. The baseline approach for OOD detection relies on the predictive uncertainty of DNNs.