LAP: An Attention-Based Module for Concept Based Self-Interpretation and Knowledge Injection in Convolutional Neural Networks

Despite the state-of-the-art performance of deep convolutional neural networks, they are susceptible to bias and malfunction in unseen situations. Moreover, the complex computation behind their reasoning is not human-understandable to develop trust. External explainer methods have tried to interpret network decisions in a human-understandable way, but they are accused of fallacies due to their assumptions and simplifications. On the other side, the inherent self-interpretability of models, while being more robust to the mentioned fallacies, cannot be applied to the already trained models. In this work, we propose a new attentionbased pooling layer, called Local Attention Pooling (LAP), that accomplishes self-interpretability and the possibility for knowledge injection without performance loss. The module is easily pluggable into any convolutional neural network, even the already trained ones. We have defined a weakly supervised training scheme to learn the distinguishing features in decision-making without depending on experts' annotations. We verified our claims by evaluating several LAP-extended models on two datasets, including ImageNet. The proposed framework offers more valid human-understandable and faithful-to-the-model interpretations than the commonly used white-box explainer methods. Nowadays, Artificial Intelligence (AI) has entered into real-life applications like clinical computer-aided decision systems, medical diagnosis, and autonomous car driving. These critical applications are concerned about whether AI models are trustable and whether their decisions are valid [41]. Deep Neural Networks (DNNs), one of the most successful AI models, make decisions using complex computations humans do not understand. They are trained end-to-end and are susceptible to learning detours and biases of the dataset rather than the actual concepts and reasons. Since AI has become responsible for making decisions in areas interfering with human rights and ethics, governments have started to make laws about its usage. For example, the European Union has adopted new regulations that enable users to demand an explanation of an algorithmic decision that has affected them [14]. This has strengthened the urge for DNNs to explain themselves. Explaining DNNs has other virtues besides verifying decisions, bias detection, developing trust, and compliance to legislation [5]; it can help diagnose the model. Also, knowledge can be discovered from the models with superior-than-human performance to enrich human knowledge [9]. In recent years, there have been many attempts to explain and interpret DNNs' decisions.