A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise

The excess risk of a M-estimator is a fundamental quantity of the theory of statistical learning. Consequently, a general theory of rates of convergence as been developed in the nineties and early 2000 ([Mas07, Kol11]). However, it has been recently identified that some theoretical descriptions of learning procedures need finer controls than those brought by the classical upper bounds of the excess risk. In this case, the derivation of concentration inequalities for the excess risk is a new and exiting axis of research, of particular importance for obtaining satisfying oracle inequalities in various contexts, especially linked to high dimension. In the field of model selection, it has been indeed remarked that such concentration inequalities allow to discuss the non-asymptotic optimality of model selection procedures ([BM07, AM09, Ler11, Sau12]).