Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making

Throughout its evolution since the 1950s, Artificial Intelligence (AI) has experienced both periods of growth and decline, known as AI springs and AI winters. However, advancements in computer hardware technology and enhanced data availability have paved the way for increased AI applications across a variety of domains, including manufacturing, healthcare, finance, management, transportation, security, education, military, and legal practice in recent years [1, 2, 3]. Artificial Neural Networks (ANNs), especially Deep Neural Networks (DNNs), demonstrated outstanding performance when applied to different tasks, including optimization, pattern recognition, data trends identification, forecasting, prediction tasks and even in query processing[4, 5, 3, 6]. However, the complex, non-linear, and multilayered architecture of these models makes the internal process and the reasoning behind such outcomes challenging to understand by the end user, turning them into "black box" models [7, 8, 9]. Deep Neural Networks (DNNs) are an example of black-box models that are frequently used in Natural Language Processing (NLP). These models are often opaque, which means it can be challenging for users to comprehend how these models derive specific predictions or decisions. The lack of transparency in deep learning models can create a lack of confidence in their outputs [10]. This absence of transparency can be particularly worrying in applications where the models' decisions carry significant consequences, such as healthcare, finance, or the criminal justice system [11].