Artificial Intelligence Risks: Automation Bias

Biased data and algorithms have been identified as significant ethical and safety concerns with artificial intelligence (AI); however, another type of bias also raises concern -- automation bias. Humans, by nature, are vulnerable to cognitive errors resulting from knowledge deficits, faulty heuristics, and affective influences. In healthcare, these cognitive missteps are known to contribute to medical errors and patient harm, particularly in relation to delayed and incorrect diagnoses. When AI is incorporated into clinical practice, healthcare providers might be susceptible to automation bias, which occurs when "clinicians accept the guidance of an automated system and cease searching for confirmatory evidence . . . Similarly, clinicians who use generally reliable technology systems might become complacent and miss potential errors, particularly if they are pressed for time or carrying a heavy workload.