Robust Conformal Outlier Detection under Contaminated Reference Data

This paper studies the problem of outlier detection: given a reference dataset (e.g., a collection of legitimate financial transactions) and an unlabeled test point (a new transaction), our goal is to determine whether the test point is an outlier (a fraudulent transaction) by assessing its deviation from the reference data distribution. Naturally, we aim to maximize the detection of outliers by harnessing the capabilities of complex machine learning (ML) models. However, these models typically lack type-I error rate control, potentially resulting in unreliable detections. In our running example, the type-I error is the probability of falsely flagging a legitimate transaction as fraudulent. As such, uncontrolled error rates can lead to costly unnecessary investigations of legitimate transactions and negatively impact customer experience. The broad need for reliable ML systems has sparked a surge of interest in conformal prediction--a versatile framework that can provide statistical guarantees for any "black-box" predictive model [38]. This framework formulates the outlier detection task as a statistical test, where the null hypothesis is that the new data point is not an outlier [23, 5]. To derive a decision rule guaranteeing type-I error control, conformal inference relies on a reference (calibration) set of inlier data points. These points are assumed to be sampled i.i.d.