MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation

Abstract--Deep neural networks (DNNs) have made significant strides in Natural Language Processing (NLP), yet their interpretability remains elusive, particularly when evaluating their intricate decision-making processes. Traditional methods often rely on post-hoc interpretations, such as saliency maps or feature visualization, which might not be directly applicable to the discrete nature of word data in NLP . Addressing this, we introduce the Model-agnostic Saliency Estimation (MASE) framework. MASE offers local explanations for text-based predictive models without necessitating in-depth knowledge of a model's internal architecture. By leveraging Normalized Linear Gaussian Perturbations (NLGP) on the embedding layer instead of raw word inputs, MASE efficiently estimates input saliency. Our results indicate MASE's superiority over other model-agnostic interpretation methods, especially in terms of Delta Accuracy, positioning it as a promising tool for elucidating the operations of text-based models in NLP . Deep learning models are becoming increasingly prevalent in Natural Language Processing (NLP) systems. V arious efficient deep neural networks (DNNs) have been proposed, ranging from classic models such as the Recurrent Neural Network (RNN) [1] and Long-Short Term Memory (LSTM) [2] to more recent Bidirectional Encoder Representations from Transformers (BERT) [3]. Although these DNNs have shown great success in natural language processing tasks, their lack of explain-ability and/or interpretability remains a major weakness when comparing and deploying these black-box models for NLP tasks such as text classification and resume recommendation. Moreover, as NLP models become increasingly complex with millions or even billions of parameters, explaining the precise decision-making process of a DNN is becoming increasingly challenging.