Noise-Adaptive Conformal Classification with Marginal Coverage

Conformal inference seeks rigorous uncertainty quantification for the predictions of any black-box machine learning model, without requiring parametric assumptions (Vovk et al., 2005). In classification, these methods aim to construct a prediction set for the label of a new test point while guaranteeing a specified coverage level. The split-conformal approach achieves this by leveraging residuals (or non-conformity scores) from a pre-trained model applied to an independent calibration data set, assuming exchangeability with the test data. Perfect exchangeability, however, may not always hold in practice, due for example to possible distribution shifts between the available data and the future test points of interest, creating a need to relax the assumptions underlying conformal inference (Barber et al., 2023).