Prediction Intervals for Machine Learning

A prediction interval is calculated as some combination of the estimated variance of the model and the variance of the outcome variable. Prediction intervals are easy to describe, but difficult to calculate in practice. In simple cases like linear regression, we can estimate the confidence interval directly. In the cases of nonlinear regression algorithms, such as artificial neural networks, it is a lot more challenging and requires the choice and implementation of specialized techniques. General techniques such as the bootstrap resampling method can be used, but are computationally expensive to calculate. The paper "A Comprehensive Review of Neural Network-based Prediction Intervals and New Advances" provides a reasonably recent study of prediction intervals for nonlinear models in the context of neural networks.