Calibrated Explanations: with Uncertainty Information and Counterfactuals

Predictive models used for AI-based decision support are generally not designed for transparency. Although they operate in critical situations such as, e.g., medicine or defence, they are limited to only presenting a probable outcome (David Gunning, 2017; Ribeiro et al., 2016), which can lead to either misuse (based on user reliance being higher than appropriate) or disuse (due to users having less reliance than appropriate) (Alvarado-Valencia & Barrero, 2014; Buçinca et al., 2020). Due to the lack of transparency, predictions from this type of model often require an explanation. In explainable artificial intelligence (XAI), the goal is to create methods that help human users identify when to trust a prediction and when not to, such as an erroneous prediction in a medical diagnosis (Marx et al., 2023). An explanation should reveal the strengths and weaknesses of the underlying model to communicate how they will behave in the future (David Gunning, 2017; Dimanov et al., 2020). There are two main categories of explanations: local explanations, which present information about the reasons for individual predictions, and global explanations, which provide information about the general behaviour of the model (Guidotti et al., 2018b; Moradi & Samwald, 2021; Martens & Foster, 2014).