Established approaches to assuring safety-critical systems and software are difficult to apply to systems employing machine learning (ML). In many cases, ML is used on ill-defined problems, e.g. optimising sepsis treatment, where there is no clear, pre-defined specification against which to assess validity. This problem is exacerbated by the "opaque" nature of ML where the learnt model is not amenable to human scrutiny. Explainable AI methods have been proposed to tackle this issue by producing human-interpretable representations of ML models which can help users to gain confidence and build trust in the ML system. However, there is not much work explicitly investigating the role of explainability for safety assurance in the context of ML development. This paper identifies ways in which explainable AI methods can contribute to safety assurance of ML-based systems. It then uses a concrete ML-based clinical decision support system, concerning weaning of patients from mechanical ventilation, to demonstrate how explainable AI methods can be employed to produce evidence to support safety assurance. The results are also represented in a safety argument to show where, and in what way, explainable AI methods can contribute to a safety case. Overall, we conclude that explainable AI methods have a valuable role in safety assurance of ML-based systems in healthcare but that they are not sufficient in themselves to assure safety.

Abstract argumentation provides us with methods such as gradual and Dung semantics with which to evaluate arguments after potential attacks by other arguments. Some of these methods can take intrinsic strengths of arguments as input, with which to modulate the effects of attacks between arguments. Coming from abstract argumentation, these methods look only at the relations between arguments and not at the structure of the arguments themselves. In structured argumentation the way an argument is constructed, by chaining inference rules starting from premises, is taken into consideration. In this paper we study methods for assigning an argument its intrinsic strength, based on the strengths of the premises and inference rules used to form said argument. We first define a set of principles, which are properties that strength assigning methods might satisfy. We then propose two such methods and analyse which principles they satisfy. Finally, we present a generalised system for creating novel strength assigning methods and speak to the properties of this system regarding the proposed principles.

One of the biggest challenges of machine learning and artificial intelligence is their inability to explain their decision to the users. This black box in AI renders the system largely impenetrable, making it difficult for scientists and researchers to understand why a certain system is behaving the way it is. In recent years, a new branch of explainable AI (XAI) has emerged, which the researchers are actively pursuing to establish user-friendly AI. That said, how AI explanations are perceived is highly dependent on a person's background in AI. A new study named "The Who in Explainable AI: How AI Background Shapes Perceptions of AI Explanations", argues that AI background influences each group's interpretations and that these differences exist through the lens of appropriation and cognitive heuristics.

Natural language free-text explanation generation is an efficient approach to train explainable language processing models for commonsense-knowledge-requiring tasks. The most predominant form of these models is the explain-then-predict (EtP) structure, which first generates explanations and uses them for making decisions. The performance of EtP models is highly dependent on that of the explainer by the nature of their structure. Therefore, large-sized explanation data are required to train a good explainer model. However, annotating explanations is expensive. Also, recent works reveal that free-text explanations might not convey sufficient information for decision making. These facts cast doubts on the effectiveness of EtP models. In this paper, we argue that the predict-then-explain (PtE) architecture is a more efficient approach in terms of the modelling perspective. Our main contribution is twofold. First, we show that the PtE structure is the most data-efficient approach when explanation data are lacking. Second, we reveal that the PtE structure is always more training-efficient than the EtP structure. We also provide experimental results that confirm the theoretical advantages.

Machine Learning (ML)-based network intrusion detection systems bring many benefits for enhancing the cybersecurity posture of an organisation. Many systems have been designed and developed in the research community, often achieving a close to perfect detection rate when evaluated using synthetic datasets. However, the high number of academic research has not often translated into practical deployments. There are several causes contributing towards the wide gap between research and production, such as the limited ability of comprehensive evaluation of ML models and lack of understanding of internal ML operations. This paper tightens the gap by evaluating the generalisability of a common feature set to different network environments and attack scenarios. Therefore, two feature sets (NetFlow and CICFlowMeter) have been evaluated in terms of detection accuracy across three key datasets, i.e., CSE-CIC-IDS2018, BoT-IoT, and ToN-IoT. The results show the superiority of the NetFlow feature set in enhancing the ML models detection accuracy of various network attacks. In addition, due to the complexity of the learning models, SHapley Additive exPlanations (SHAP), an explainable AI methodology, has been adopted to explain and interpret the classification decisions of ML models. The Shapley values of two common feature sets have been analysed across multiple datasets to determine the influence contributed by each feature towards the final ML prediction.

Researchers from the National University of Singapore have concluded that the more explainable AI becomes, the easier it will become to circumvent vital privacy features in machine learning systems. They also found that even when a model is not explainable, it's possible to use explanations of similar models to'decode' sensitive data in the non-explainable model. The research, titled Exploiting Explanations for Model Inversion Attacks, highlights the risks of using the'accidental' opacity of the way neural networks function as if this was a by-design security feature – not least because a wave of new global initiatives, including the European Union's draft AI regulations, are characterizing explainable AI (XAI) as a prerequisite for the eventual normalization of machine learning in society. In the research, an actual identity is successfully reconstructed from supposedly anonymous data relating to facial expressions, through the exploitation of multiple explanations of the machine learning system. 'Explainable artificial intelligence (XAI) provides more information to help users to understand model decisions, yet this additional knowledge exposes additional risks for privacy attacks.

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. As AI systems increasingly inform decision-making in health care, finance, law, and criminal justice, they need to provide justifications for their behavior that humans can understand. The field of "explainable AI" has gained momentum as regulators turn a critical eye toward black-box AI systems -- and their creators. But how a person's background can shape perceptions of AI explanations is a question that remains underexplored. A new study coauthored by researchers at Cornell University, IBM, and the Georgia Institute of Technology aims to shed light on the intersection of interpretability and explainable AI.

I recently fell in love with Explainable AI (XAI). XAI is a set of methods aimed at making increasingly complex machine learning (ML) models understandable by humans. XAI could help bridge the gap between AI and humans. That is very much needed as the gap is widening. Machine learning is proving incredibly successful in tackling problems from cancer diagnostics to fraud detection.

The biggest shortcoming of many machine learning models and neural networks is their "blackbox" nature. Which feature was most influential in this predicted output that we got for an instance? XAI which stands for Explainable Artificial Intelligence is the area of study that tries to tackle this blackbox issue of models.

This extends the reach of Boolean logic by enabling a variety of applications that have been explored over the decades. The existential quantification of literals (variable states) and its applications have also been studied in the literature. In this paper, we complement this by studying universal literal quantification and its applications, particularly to explainable AI. We also provide a novel semantics for quantification, discuss the interplay between variable/literal and existential/universal quantification. We further identify some classes of Boolean formulas and circuits on which quantification can be done efficiently. Literal quantification is more fine-grained than variable quantification as the latter can be defined in terms of the former. This leads to a refinement of quantified Boolean logic with literal quantification as its primitive.