On the universal distribution of the coverage in split conformal prediction

Conformal prediction [1, 2, 3, 4, 5, 6], a technique developed to address the confidence in the forecasts made by general predictive models, is quickly moving the field of machine learning [7, 8, 9, 10] from a period dominated by point predictions, to a new stage in which inferences about the future are summarized by prediction sets with statistical guarantees. Several features make conformal prediction appealing for use with contemporary machine learning algorithms: it is universal (distribution-free), able to handle high-dimensional data, model agnostic, and its properties hold for finite samples. This paper strengthens the universal properties of the most readily applicable variation on the conformal prediction idea: the split conformal prediction algorithm [2, 5], whose implementation attains a good balance between the predictive goals and the computational complexity of the procedure. In a regression context, the main results of the paper are the identification for exchangeable data of the exact distribution of the coverage of prediction sets for a finite horizon of future observables (future coverage, for short), and the determination of the exact distribution of its almost sure limit when the horizon tends to infinity.