Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution

It was first introduced to data analysis by Hoerl (1959) and later formulated in Hoerl and Kennard (1970b,a) for providing a robust solution to some of the persistent challenges encountered in traditional linear regression techniques; see Hoerl (1985) for a nice review. Emerging as a fundamental technique in predictive modeling, ridge regression addresses issues such as multicollinearity and overfitting, which commonly afflict predictive models dealing with high-dimensional data. Since its inception, ridge regression's practical adoption persists due to its superior performance over the least-squares estimator in various scenarios, evident in applications across neuroscience, chemistry, biology, and economics; see Leonard et al. (2023), Zahrt et al. (2019), Otwinowski and Plotkin (2014), Giannone et al. (2021), and Abadie and Kasy (2019), among others, underscoring its empirical effectiveness. From a shrinkage perspective, the ridge estimator also dominates the least-squares solutions in the sense that its mean-squared errors (MSEs) can be smaller, which provides a reasonable explanation on the empirical effectiveness of ridge estimators. See Theobald (1974), Athey and Imbens (2019), Hastie (2020), Hansen (2022a), and a comprehensive introduction to ridge regression in van Wieringen (2023). The ridge estimator offers a closed-form expression that simplifies both theoretical and empirical analyses. It aligns with the dense modeling techniques of Giannone et al. (2021), which acknowledge the potential significance of all explanatory variables for prediction. Empirical studies, such as those in Giannone et al. (2021), indicate that dense models generally tend to outperform the sparse ones in out-of-sample economic prediction performance. Similarly, Abadie and Kasy (2019) find that the ridge estimators dominate the lasso and the pre-testing estimators in terms of the risks when the effects of different predictors on the dependent variable are "smoothly distributed".