We describe a fast computation method for leave-one-out cross-validation (LOOCV) for $k$-nearest neighbours ($k$-NN) regression. We show that, under a tie-breaking condition for nearest neighbours, the LOOCV estimate of the mean square error for $k$-NN regression is identical to the mean square error of $(k+1)$-NN regression evaluated on the training data, multiplied by the scaling factor $(k+1)^2/k^2$. Therefore, to compute the LOOCV score, one only needs to fit $(k+1)$-NN regression only once, and does not need to repeat training-validation of $k$-NN regression for the number of training data. Numerical experiments confirm the validity of the fast computation method.

Cross-validation is a technique for evaluating a machine learning model and testing its performance. Cross-validation is a technique used to evaluate the performance of a machine learning model by training it on different subsets of the data and testing it on the remaining subset. Cross-validation is also known as rotation estimation or out-of-sample testing. Rotation estimation refers to the process of rotating, or splitting, the data into different subsets. Simply put, in the process of cross-validation, the original data sample is randomly divided into several subsets.

It's one of the technique in which we implement KFold cross-validation, where k is equal to n i.e the number of observations in the data. Thus, every single point will be used in a validation set, we will create n models, for n-observations in the data. Each point/sample is used once as a test set while the remaining data/samples form the training set. The scikit-learn Python machine learning library provides an implementation of the LOOCV via the LeaveOneOut class using Leave-One-Out cross-validator.