Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

Explainable AI (XAI) methods are used in order to improve the interpretability of these complex "black box" models, thereby increasing transparency and enabling informed decision-making (Guidotti et al, 2018). Despite this, methods to assess the quality of explanations generated by such explainable methods are so far under-explored. In particular, functionallygrounded evaluation methods, which measure the inherent ability of explainable methods in a given situation, are often specific to a particular type of dataset or explainable method. A key measure of functionally-grounded explanation fitness is explanation fidelity, which assesses the correctness and completeness of the explanation with respect to the underlying black box predictive model (Zhou et al, 2021). Evaluations of fidelity in literature can generally be classified as one of the following: external fidelity evaluation, which assesses how well the prediction of the underlying model and the explanation agree, and internal fidelity, which assesses how well the explanation matches the decision-making processes of the underlying model (Messalas et al, 2019). While methods to evaluate external fidelity are relatively common in literature (Guidotti et al, 2019; Lakkaraju et al, 2016; Ming et al, 2019; Shankaranarayana and Runje, 2019), evaluation methods to evaluate internal fidelity using black box models are generally limited to text and image data, rather than tabular (Du et al, 2019; Fong and Vedaldi, 2017; Nguyen, 2018; Samek et al, 2017). In this paper, weproposeanovelevaluation method based onathree phase approach:(1) the creation of a fully transparent, inherently interpretable white box model, and evaluation of explanations against this model; (2) the usage of the white box as a proxy to refine and improve the evaluation of explanations generated by a black box model; and (3) test the fidelity of explanations for a black box model using the refined method from the second phase. The main contributions of this work are as follows: 1.