cp interval
Conformal Prediction Intervals with Tail-Specific Guarantees
This paper extends classical conformal frameworks for constructing prediction intervals with global marginal coverage $1-α$ to intervals that provide explicitly calibrated guarantees for the upper and lower tails separately. Focusing on split conformal prediction, we first construct lower and upper one-sided conformal intervals that achieve marginal validity, and then derive the induced two-sided interval by intersection. Theoretical results prove both tail-specific and global marginal coverage of the induced two-sided interval. Results are presented first for the exchangeable setting, where coverage has finite-sample guarantees, and then for non-exchangeable data, where guarantees are asymptotic. Simulation studies show that the proposed approach achieves improved directional calibration relative to classical two-sided intervals, especially relevant in skewed data. Finally, the benefit of the proposed framework is showcased in a financial application, where one aims for return maximization while seeking strict control on the left tail.
Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervals
Idrissi, Marouane Il, Machado, Agathe Fernandes, Gallic, Ewen, Charpentier, Arthur
Cooperative game theory methods, notably Shapley values, have significantly enhanced machine learning (ML) interpretability. However, existing explainable AI (XAI) frameworks mainly attribute average model predictions, overlooking predictive uncertainty. This work addresses that gap by proposing a novel, model-agnostic uncertainty attribution (UA) method grounded in conformal prediction (CP). By defining cooperative games where CP interval properties-such as width and bounds-serve as value functions, we systematically attribute predictive uncertainty to input features. Extending beyond the traditional Shapley values, we use the richer class of Harsanyi allocations, and in particular the proportional Shapley values, which distribute attribution proportionally to feature importance. We propose a Monte Carlo approximation method with robust statistical guarantees to address computational feasibility, significantly improving runtime efficiency. Our comprehensive experiments on synthetic benchmarks and real-world datasets demonstrate the practical utility and interpretative depth of our approach. By combining cooperative game theory and conformal prediction, we offer a rigorous, flexible toolkit for understanding and communicating predictive uncertainty in high-stakes ML applications.
Export Reviews, Discussions, Author Feedback and Meta-Reviews
We like to thank the reviewers for their positive feedback! General comments: - Although we agree that the assumption of the Plackett-Luce model (as a generalization of the Bradley-Terry model) may appear restrictive and will certainly not be satisfied in all practical applications, we like to emphasize that the PL model, in addition to the Mallows model, is the standard model in the statistics of rank data and widely used in many fields of applied statistics, e.g., voting and discrete choice theory in economics -- its status in these fields is comparable to the status of the Gaussian distribution for real-valued data. Therefore, we are convinced that studying the dueling bandits problem under this assumption is a worthwhile endeavor. In this regard, we also like to mention that the PL model has already been studied in the context of other preference learning problems as well (for example, see papers at ICML 2009 and 2010). Rev 1: The confidence intervals in our paper are derived from Hoeffding's inequality in a standard way.
Conformal Prediction for Causal Effects of Continuous Treatments
Schröder, Maresa, Frauen, Dennis, Schweisthal, Jonas, Heß, Konstantin, Melnychuk, Valentyn, Feuerriegel, Stefan
Uncertainty quantification of causal effects is crucial for safety-critical applications such as personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to model-agnostic finite-sample guarantees. Yet, existing methods for conformal prediction of causal effects are limited to binary/discrete treatments and make highly restrictive assumptions such as known propensity scores. In this work, we provide a novel conformal prediction method for potential outcomes of continuous treatments. We account for the additional uncertainty introduced through propensity estimation so that our conformal prediction intervals are valid even if the propensity score is unknown. Our contributions are three-fold: (1) We derive finite-sample prediction intervals for potential outcomes of continuous treatments. (2) We provide an algorithm for calculating the derived intervals. (3) We demonstrate the effectiveness of the conformal prediction intervals in experiments on synthetic and real-world datasets. To the best of our knowledge, we are the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data.