Explanation & Argumentation
Local Explanations for Clinical Search Engine results
Contempré, Edeline, Szlávik, Zoltán, Mohammadi, Majid, Velazquez, Erick, Teije, Annette ten, Tiddi, Ilaria
Health care professionals rely on treatment search engines to efficiently find adequate clinical trials and early access programs for their patients. However, doctors lose trust in the system if its underlying processes are unclear and unexplained. In this paper, a model-agnostic explainable method is developed to provide users with further information regarding the reasons why a clinical trial is retrieved in response to a query. To accomplish this, the engine generates features from clinical trials using by using a knowledge graph, clinical trial data and additional medical resources. and a crowd-sourcing methodology is used to determine their importance. Grounded on the proposed methodology, the rationale behind retrieving the clinical trials is explained in layman's terms so that healthcare processionals can effortlessly perceive them. In addition, we compute an explainability score for each of the retrieved items, according to which the items can be ranked. The experiments validated by medical professionals suggest that the proposed methodology induces trust in targeted as well as in non-targeted users, and provide them with reliable explanations and ranking of retrieved items.
A Formalisation of Abstract Argumentation in Higher-Order Logic
Steen, Alexander, Fuenmayor, David
We present an approach for representing abstract argumentation frameworks based on an encoding into classical higher-order logic. This provides a uniform framework for computer-assisted assessment of abstract argumentation frameworks using interactive and automated reasoning tools. This enables the formal analysis and verification of meta-theoretical properties as well as the flexible generation of extensions and labellings with respect to well-known argumentation semantics.
Improving Users' Mental Model with Attention-directed Counterfactual Edits
Alipour, Kamran, Ray, Arijit, Lin, Xiao, Cogswell, Michael, Schulze, Jurgen P., Yao, Yi, Burachas, Giedrius T.
In the domain of Visual Question Answering (VQA), studies have shown improvement in users' mental model of the VQA system when they are exposed to examples of how these systems answer certain Image-Question (IQ) pairs. In this work, we show that showing controlled counterfactual image-question examples are more effective at improving the mental model of users as compared to simply showing random examples. We compare a generative approach and a retrieval-based approach to show counterfactual examples. We use recent advances in generative adversarial networks (GANs) to generate counterfactual images by deleting and inpainting certain regions of interest in the image. We then expose users to changes in the VQA system's answer on those altered images. To select the region of interest for inpainting, we experiment with using both human-annotated attention maps and a fully automatic method that uses the VQA system's attention values. Finally, we test the user's mental model by asking them to predict the model's performance on a test counterfactual image. We note an overall improvement in users' accuracy to predict answer change when shown counterfactual explanations. While realistic retrieved counterfactuals obviously are the most effective at improving the mental model, we show that a generative approach can also be equally effective.
CNN's Dr. Sanjay Gupta explains appearance on Joe Rogan podcast: 'I needed to go into the lion's den'
In media news today, Katie Couric admits she protected Ruth Bader Ginsburg by editing out remarks on anthem kneelers, a former Obama ethics official slams the Biden White House for avoiding questions on Hunter Biden's artwork, and Facebook says it will treat journalist and activists as public figures CNN chief medical correspondent Dr. Sanjay Gupta sought to explain the reasoning behind his appearance on Joe Rogan's podcast "The Joe Rogan Experience" this week, claiming he felt he "needed to go into the lion's den" to communicate to people about public health. Gupta faced intense criticism stemming from his appearance on the show, in which he admitted that CNN shouldn't have referred to Rogan's use of the drug ivermectin to treat the coronavirus as him using "horse dewormer." CNN did not immediately respond when asked if Gupta was forced to justify the appearance in his Wednesday piece following the backlash. In the piece titled "Why Joe Rogan and I sat down and talked -- for more than 3 hours," Gupta detailed his conversation with Rogan, including his "futile" attempt to convince the popular radio host to take the coronavirus vaccine, and compared it to being in a mixed martial arts (MMA) bout. "I realized that if I was serious about trying to communicate public health, I needed to go to a less comfortable place. I needed to go into the lion's den and accept an invitation to sit down with Joe Rogan for more than three hours," Gupta wrote before admitting that many of his friends advised him not to accept Rogan's invitation.
On Quantifying Literals in Boolean Logic and its Applications to Explainable AI
Darwiche, Adnan (UCLA) | Marquis, Pierre
Quantified Boolean logic results from adding operators to Boolean logic for existentially and universally quantifying variables. This extends the reach of Boolean logic by enabling a variety of applications that have been explored over the decades. The existential quantification of literals (variable states) and its applications have also been studied in the literature. In this paper, we complement this by introducing and studying universal literal quantification and its applications, particularly to explainable AI. We also provide a novel semantics for quantification, discuss the interplay between variable/literal and existential/universal quantification, and identify some classes of Boolean formulas and circuits on which quantification can be done efficiently. Literal quantification is more fine-grained than variable quantification as the latter can be defined in terms of the former, leading to a refinement of quantified Boolean logic with literal quantification as its primitive.
Explainable Fact-checking through Question Answering
Yang, Jing, Vega-Oliveros, Didier, Seibt, Taís, Rocha, Anderson
Misleading or false information has been creating chaos in some places around the world. To mitigate this issue, many researchers have proposed automated fact-checking methods to fight the spread of fake news. However, most methods cannot explain the reasoning behind their decisions, failing to build trust between machines and humans using such technology. Trust is essential for fact-checking to be applied in the real world. Here, we address fact-checking explainability through question answering. In particular, we propose generating questions and answers from claims and answering the same questions from evidence. We also propose an answer comparison model with an attention mechanism attached to each question. Leveraging question answering as a proxy, we break down automated fact-checking into several steps -- this separation aids models' explainability as it allows for more detailed analysis of their decision-making processes. Experimental results show that the proposed model can achieve state-of-the-art performance while providing reasonable explainable capabilities.
A Framework for Rationale Extraction for Deep QA models
Ramnath, Sahana, Nema, Preksha, Sahni, Deep, Khapra, Mitesh M.
As neural-network-based QA models become deeper and more complex, there is a demand for robust frameworks which can access a model's rationale for its prediction. Current techniques that provide insights on a model's working are either dependent on adversarial datasets or are proposing models with explicit explanation generation components. These techniques are time-consuming and challenging to extend to existing models and new datasets. In this work, we use `Integrated Gradients' to extract rationale for existing state-of-the-art models in the task of Reading Comprehension based Question Answering (RCQA). On detailed analysis and comparison with collected human rationales, we find that though ~40-80% words of extracted rationale coincide with the human rationale (precision), only 6-19% of human rationale is present in the extracted rationale (recall).
Explaining Reward Functions to Humans for Better Human-Robot Collaboration
Sanneman, Lindsay, Shah, Julie
Explainable AI techniques that describe agent reward functions can enhance human-robot collaboration in a variety of settings. One context where human understanding of agent reward functions is particularly beneficial is in the value alignment setting. In the value alignment context, an agent aims to infer a human's reward function through interaction so that it can assist the human with their tasks. If the human can understand where gaps exist in the agent's reward understanding, they will be able to teach more efficiently and effectively, leading to quicker human-agent team performance improvements. In order to support human collaborators in the value alignment setting and similar contexts, it is first important to understand the effectiveness of different reward explanation techniques in a variety of domains. In this paper, we introduce a categorization of information modalities for reward explanation techniques, suggest a suite of assessment techniques for human reward understanding, and introduce four axes of domain complexity. We then propose an experiment to study the relative efficacy of a broad set of reward explanation techniques covering multiple modalities of information in a set of domains of varying complexity.
Interactively Generating Explanations for Transformer Language Models
Schramowski, Patrick, Friedrich, Felix, Tauchmann, Christopher, Kersting, Kristian
Transformer language models are state-of-the-art in a multitude of NLP tasks. Despite these successes, their opaqueness remains problematic. Recent methods aiming to provide interpretability and explainability to black-box models primarily focus on post-hoc explanations of (sometimes spurious) input-output correlations. Instead, we emphasize using prototype networks directly incorporated into the model architecture and hence explain the reasoning process behind the network's decisions. Moreover, while our architecture performs on par with several language models, it enables one to learn from user interactions. This not only offers a better understanding of language models but uses human capabilities to incorporate knowledge outside of the rigid range of purely data-driven approaches.