Overfitting Can Be Harmless for Basis Pursuit, But Only to a Degree

Recently, there have been significant interests in studying the so-called "doubledescent" of the generalization error of linear regression models under the overparameterized and overfitting regime, with the hope that such analysis may provide the first step towards understanding why overparameterized deep neural networks (DNN) still generalize well.