Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension Moritz Haas 1 David Holzmüller

In this paper, we show that the smoothness of the estimators, and not the dimension, is the key: benign overfitting is possible if and only if the estimator's derivatives are large enough.