Universal Prediction Band via Semi-Definite Programming

A frequent criticism from the statistics community to modern machine learning is the lack of rigorous uncertainty quantification. Instead, the machine learning community would argue that conventional uncertainty quantification based on idealized distributional assumptions may be too restrictive for real data. Nevertheless, without a doubt, uncertainty quantification for predictive modeling is essential to statistics, learning theory, and econometrics. This paper will resolve the above inference dilemma by introducing a new method with provable uncertainty quantification via semi-definite programming. The proposed method learns a dataadaptive, heteroskedastic prediction band that is: (a) universally applicable without strong distributional assumptions, (b) with desirable theoretical coverage with or without any user-specified predictive model, and (c) easy to implement via standard convex programs (when used in conjunction with a wide range of positive-definite kernels). Let (x, y) X R be the covariates and response data pair drawn from an unknown distribution P.