Distilling neural networks into skipgram-level decision lists

Murdoch and Szlam Understanding and explaining decisions of complex (2017) explain long short term memory networks models such as neural networks has recently (LSTMs) (Hochreiter and Schmidhuber, 1997) by gained a lot of attention for engendering trust in means of ngram rules, but their rules are limited these models, improving them, and understanding to presence of single ngrams and do not capture them better (Montavon et al., 2018; Alishahi et al., interaction between ngrams in text. To learn explanation 2019; Belinkov and Glass, 2019). Several previous rules for RNNs while overcoming the studies developing interpretability techniques provide limitations of the previous approaches, we have the a set of input features or segments that are the following contributions in the paper: most salient for the model output. Approaches such as input perturbation and gradient computation are 1. We induce explanation rules over important popular for this purpose (Ancona et al., 2018; Arras skipgrams in text, while ensuring that these et al., 2019). A drawback of these approaches rules generalize to unseen data. To this end, is the lack of information about interaction between we quantify skipgram importance in LSTMs different features. While heatmaps (Li et al., by first pooling gradients across embedding 2016b,a; Arras et al., 2017) and partial dependence dimensions to compute word importance, and plots (Lundberg and Lee, 2017) are popularly used, thereby aggregating them into skipgram importance. Research conducted while at CLiPS.