Towards a Unified Evaluation of Explanation Methods without Ground Truth

This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To this end, we design four metrics to evaluate explanation results without ground-truth explanations. Our metrics can be broadly applied to nine benchmark methods of interpreting neural networks, which provides new insights of explanation methods. Nowadays, many methods are proposed to explain the feature representations of a deep neural network (DNN) in a post-hoc manner. However, some methods usually pursue attribution maps which look reasonable from the perspective of human users, instead of objectively reflecting the information processing in the DNN. A trustworthy evaluation of the objectiveness of attribution maps is crucial for the development of deep learning and proposes significant challenges to state-of-the-art algorithms. Existing metrics (Y ang & Kim, 2019; Arras et al., 2019) of evaluating explanation methods have certain shortcomings. Issue 1, evaluation of the accuracy of a DNN null evaluation of the objectiveness of attribution maps: Some methods only evaluate whether the visualized attribution map looks reasonable to human users, instead of examining whether an attribution map objectively reflects the truth of a DNN. For example, they added an irrelevant object into the image. Pixels from the irrelevant object are expected to be assigned with zero attributions.