EMAP: Explanation by Minimal Adversarial Perturbation

These methods generally return either a weighting or subset of input features as an explanation of the classification of an instance. An alternative literature argues instead that counterfactual instances provide a more useable characterisation of a black box classifier's decisions. We present EMAP, a neural network based approach which returns as Explanation the Minimal Adversarial Perturbation to an instance required to cause the underlying black box model to missclassify. We show that this approach combines the two paradigms, recovering the output of feature-weighting methods in continuous feature spaces, whilst also indicating the direction in which the nearest counterfactuals can be found. Our method also provides an implicit confidence estimate in its own explanations, adding a clarity to model diagnostics other methods lack. Additionally, EMAP improves upon the speed of sampling-based methods such as LIME by an order of magnitude, allowing for model explanations in time-critical applications, or at the dataset level, where sampling-based methods are infeasible. We extend our approach to categorical features using a partitioned Gumbel layer, and demonstrate its efficacy on several standard datasets. Introduction Recent interest in explaining the output of complex machine learning models has been characterized by a wide range of approaches (Lipton 2016; Montavon, Samek, and M uller 2018). Many of these approaches are model specific; for example attempts to explain neural networks that rely on interpreting the flow of gradient information through the model (Shrikumar, Greenside, and Kundaje 2017; Olah, Mordv-intsev, and Schubert 2017; Karpathy, Johnson, and Fei-Fei 2015), or decision trees, which might be considered directly interpretable (Molnar 2019).