Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation

Algorithmic approaches to "explaining" model predictions have proliferated as machine learning methods gain in popularity. We refer interested readers to Molnar (2022) or Arrieta et al. (2020) for in-depth surveys. For the purposes of this paper, we simply note that there are many high-level approaches to explaining model predictions and we focus solely on SHAP (Lundberg and Lee (2017)). SHAP is a popular "local feature attribution" method, which means it attempts to explain a model by "scoring" input feature contributions for a specific prediction. Other common examples of local attribution methods include LIME (Ribeiro et al. (2016)), Integrated Gradients (Sundararajan et al. (2017)), and GradCAM (Selvaraju et al. (2017)). While each of these methods deserve detailed study, this paper is a thorough investigation of the statistical properties of SHAP. SHAP applies the Shapley value from game theory (Shapley (1953)) to model explanation by considering features as "players" in a cooperative game. Lundberg and Lee (2017) approximate Shapley values for each feature using a weighted least squares regression, where the regression weights are a transformation of the original Shapley value weights.