Lifted Coefficient of Determination: Fast model-free prediction intervals and likelihood-free model comparison

A common objective in statistical learning is uncertainty quantification. This task is well-studied and forms a key component of supervised and unsupervised statistical learning. In the context of regression this amounts to estimating the generalisation error, i.e., the expected mean-squared prediction error (MSPE) for unknown test observations. Usually either analytic methods for a specific model, or computational algorithms, e.g., the bootstrap or cross-validation, are employed for obtaining generalisation error estimates; see, e.g., Stone (1974), Efron & Tibshirani (1994), Chapter 7 in Hastie et al. (2017), Hastie et al. (2015), Nadeau & Bengio (1999), Wainwright (2019), Wasserman (2010). Moreover, these generalisation error estimates can be used for comparing different models, cross-validation during training, and selecting a final model from a finite collection of candidate models; see, e.g., Wood (2017), David J. Olive & Haile (2022), Das & Nason (2016).