slundberg/shap

SHAP (SHapley Additive exPlanations) explains the output of any machine learning model using expectations and Shapley values. SHAP unifies aspects of several previous methods [1-7] and represents the only possible consistent and locally accurate additive feature attribution method based on expectations (see SHAP paper for details). While SHAP values can explain the output of any machine learning model, we have developed a high-speed exact algorithm for ensemble tree methods (Tree SHAP paper). The above explanation shows features each contributing to push the model output from the base value (the average model output over the training dataset we passed) to the model output. Features pushing the prediction higher are shown in red, those pushing the prediction lower are in blue.