Goto

Collaborating Authors

 Explanation & Argumentation


survex: an R package for explaining machine learning survival models

arXiv.org Machine Learning

Summary: Due to their flexibility and superior performance, machine learning models frequently complement and outperform traditional statistical survival models. However, their widespread adoption is hindered by a lack of user-friendly tools to explain their internal operations and prediction rationales. To tackle this issue, we introduce the survex R package, which provides a cohesive framework for explaining any survival model by applying explainable artificial intelligence techniques. The capabilities of the proposed software encompass understanding and diagnosing survival models, which can lead to their improvement. By revealing insights into the decision-making process, such as variable effects and importances, survex enables the assessment of model reliability and the detection of biases. Thus, transparency and responsibility may be promoted in sensitive areas, such as biomedical research and healthcare applications.


Automatic Analysis of Substantiation in Scientific Peer Reviews

arXiv.org Artificial Intelligence

With the increasing amount of problematic peer reviews in top AI conferences, the community is urgently in need of automatic quality control measures. In this paper, we restrict our attention to substantiation -- one popular quality aspect indicating whether the claims in a review are sufficiently supported by evidence -- and provide a solution automatizing this evaluation process. To achieve this goal, we first formulate the problem as claim-evidence pair extraction in scientific peer reviews, and collect SubstanReview, the first annotated dataset for this task. SubstanReview consists of 550 reviews from NLP conferences annotated by domain experts. On the basis of this dataset, we train an argument mining system to automatically analyze the level of substantiation in peer reviews. We also perform data analysis on the SubstanReview dataset to obtain meaningful insights on peer reviewing quality in NLP conferences over recent years.


Designing Interpretable ML System to Enhance Trustworthy AI in Healthcare: A Systematic Review of the Last Decade to A Proposed Robust Framework

arXiv.org Artificial Intelligence

AI-based medical technologies, including wearables, telemedicine, LLMs, and digital care twins, significantly impact healthcare. Ensuring AI results are accurate and interpretable is crucial, especially for clinicians. This paper reviews processes and challenges of interpretable ML (IML) and explainable AI (XAI) in healthcare. Objectives include reviewing XAI processes, methods, applications, and challenges, with a focus on quality control. The IML process is classified into data pre-processing interpretability, interpretable modeling, and post-processing interpretability. The paper aims to establish the importance of robust interpretability in healthcare through experimental results, providing insights for creating communicable clinician-AI tools. Research questions, eligibility criteria, and goals were identified following PRISMA and PICO methods. PubMed, Scopus, and Web of Science were systematically searched using specific strings. The survey introduces a step-by-step roadmap for implementing XAI in clinical applications, addressing existing gaps and acknowledging XAI model limitations.


A novel post-hoc explanation comparison metric and applications

arXiv.org Artificial Intelligence

Explanatory systems make the behavior of machine learning models more transparent, but are often inconsistent. To quantify the differences between explanatory systems, this paper presents the Shreyan Distance, a novel metric based on the weighted difference between ranked feature importance lists produced by such systems. This paper uses the Shreyan Distance to compare two explanatory systems, SHAP and LIME, for both regression and classification learning tasks. Because we find that the average Shreyan Distance varies significantly between these two tasks, we conclude that consistency between explainers not only depends on inherent properties of the explainers themselves, but also the type of learning task. This paper further contributes the XAISuite library, which integrates the Shreyan distance algorithm into machine learning pipelines.


Counterfactuals in explainable AI: interview with Ulrike Kuhl

AIHub

Objective task performance, measured as development of mean number of generated aliens per round per group. In their work For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI, Ulrike Kuhl, and colleagues André Artelt and Barbara Hammer, have investigated counterfactual explanations in explainable artificial intelligence. In this interview, Ulrike tells us more about their study, and highlights some of their interesting and surprising findings. Counterfactual statements are something we as humans use every day. Phrases like "If I had gotten up earlier, I would have been on time" are counterfactuals, describing a hypothetical alternative to the current, factual state. In the realm of explainable AI (xAI), counterfactuals play a pivotal role by providing accessible and intuitive insights into the decision-making processes of AI models.


Diagnosing AI Explanation Methods with Folk Concepts of Behavior

Journal of Artificial Intelligence Research

We investigate a formalism for the conditions of a successful explanation of AI. We consider "success" to depend not only on what information the explanation contains, but also on what information the human explainee understands from it. Theory of mind literature discusses the folk concepts that humans use to understand and generalize behavior. We posit that folk concepts of behavior provide us with a "language" that humans understand behavior with. We use these folk concepts as a framework of social attribution by the human explainee--the information constructs that humans are likely to comprehend from explanations--by introducing a blueprint for an explanatory narrative (Figure 1) that explains AI behavior with these constructs. We then demonstrate that many XAI methods today can be mapped to folk concepts of behavior in a qualitative evaluation. This allows us to uncover their failure modes that prevent current methods from explaining successfully--i.e., the information constructs that are missing for any given XAI method, and whose inclusion can decrease the likelihood of misunderstanding AI behavior.


Understanding Path Planning Explanations

arXiv.org Artificial Intelligence

Abstract--Navigation is a must-have skill for any mobile robot. For the design of our user study, we will extend and formalize our approach to explain both path planning failures There is an increasing deployment of autonomous robots and deviations from the initial trajectory, i.e. to be able to in various domains [1]. Currently, we use one and accountability in their decision-making exists [2]. Navigation is a pivotal aspect of an autonomous robot create different planning failures and trajectory-contrastive decision-making spectrum. After generating explanations for the created scenarios, a key role in achieving accurate and efficient navigation in we will perturb the explanations in the following way: changing environments.


On the Interplay between Fairness and Explainability

arXiv.org Artificial Intelligence

In order to build reliable and trustworthy NLP applications, models need to be both fair across different demographics and explainable. Usually these two objectives, fairness and explainability, are optimized and/or examined independently of each other. Instead, we argue that forthcoming, trustworthy NLP systems should consider both. In this work, we perform a first study to understand how they influence each other: do fair(er) models rely on more plausible rationales? and vice versa. To this end, we conduct experiments on two English multi-class text classification datasets, BIOS and ECtHR, that provide information on gender and nationality, respectively, as well as human-annotated rationales. We fine-tune pre-trained language models with several methods for (i) bias mitigation, which aims to improve fairness; (ii) rationale extraction, which aims to produce plausible explanations. We find that bias mitigation algorithms do not always lead to fairer models. Moreover, we discover that empirical fairness and explainability are orthogonal.


Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

arXiv.org Machine Learning

Machine learning is currently undergoing an explosion in capability, popularity, and sophistication. However, one of the major barriers to widespread acceptance of machine learning (ML) is trustworthiness: most ML models operate as black boxes, their inner workings opaque and mysterious, and it can be difficult to trust their conclusions without understanding how those conclusions are reached. Explainability is therefore a key aspect of improving trustworthiness: the ability to better understand, interpret, and anticipate the behaviour of ML models. To this end, we propose SMILE, a new method that builds on previous approaches by making use of statistical distance measures to improve explainability while remaining applicable to a wide range of input data domains.


FATE in AI: Towards Algorithmic Inclusivity and Accessibility

arXiv.org Artificial Intelligence

Examples of bias and discrimination in AI applications include court decisions [1], job hiring [2], online ads [3], and many other areas prone to bias [4]. These algorithmic decisions have economic and personal implications for individuals. Therefore, Fairness, Accountability, Transparency and Ethics (FATE) in AI must be properly regulated for responsible use cases [5, 6], particularly in high-stakes domains [1, 7, 8, 9, 10, 11, 12]. Studies have shown that machine learning models can discriminate based on race and gender [13, 14, 15]. FATE in AI is intended to address the social issues caused by digital systems, but the current discourse is largely shaped by more economically developed countries (MEDC), raising concerns about neglecting local knowledge, cultural pluralism, and global fairness [16]. As AI systems become more integrated into various products [9, 10, 17, 12, 18, 19], they are a major driver of the fourth industrial revolution (4IR) and transformation [20]. Therefore, it is essential to understand the FATE-related needs of different communities, as AI affects a wide range of people. Ensuring effective transparency cannot be a one-size-fits-all approach [21], as this could disproportionately affect different communities [16, 22]. To this end, more contextualised and interdisciplinary research is needed to inform algorithmic fairness and transparency [23, 24, 25].