A comparison of some conformal quantile regression methods

Matteo Sesia 1 and Emmanuel J. Cand es 1,2 1 Department of Statistics, Stanford University 2 Department of Mathematics, Stanford University September 13, 2019 Abstract We compare two recently proposed methods that combine ideas from conformal inference and quantile regression to produce locally adaptive and marginally valid prediction intervals under sample exchangeability (Romano et al., 2019 [1]; Kivaranovic et al., 2019 [2]). First, we prove that these two approaches are asymptotically efficient in large samples, under some additional assumptions. Then we compare them empirically on simulated and real data. Our results demonstrate that the method in Romano et al. (2019) typically yields tighter prediction intervals in finite samples. Finally, we discuss how to tune these procedures by fixing the relative proportions of observations used for training and conformalization. 1 Introduction 1.1 Background and motivation Given a set of n points { (X i,Y i) } n i 1, with Y i R and X i R d, we consider the problem of constructing a prediction interval for a new point Y n 1based on the observed value of X n 1, assuming only that { (X i,Y i) } n 1 i 1 are drawn exchangeably from some common distribution P XY. There exist a vast selection of statistical and machine learning algorithms that can provide approximate answers to this question [3, 4].