In the United States, primary open-angle glaucoma (POAG) is the leading cause of blindness, especially among African American and Hispanic individuals. Deep learning has been widely used to detect POAG using fundus images as its performance is comparable to or even surpasses diagnosis by clinicians. However, human bias in clinical diagnosis may be reflected and amplified in the widely-used deep learning models, thus impacting their performance. Biases may cause (1) underdiagnosis, increasing the risks of delayed or inadequate treatment, and (2) overdiagnosis, which may increase individuals' stress, fear, well-being, and unnecessary/costly treatment. In this study, we examined the underdiagnosis and overdiagnosis when applying deep learning in POAG detection based on the Ocular Hypertension Treatment Study (OHTS) from 22 centers across 16 states in the United States. Our results show that the widely-used deep learning model can underdiagnose or overdiagnose underserved populations. The most underdiagnosed group is female younger (< 60 yrs) group, and the most overdiagnosed group is Black older (>=60 yrs) group. Biased diagnosis through traditional deep learning methods may delay disease detection, treatment and create burdens among under-served populations, thereby, raising ethical concerns about using deep learning models in ophthalmology clinics.

Google's entry into health diagnostics has alarmed health experts who fear a new artificial intelligence tool to identify skin conditions could lead to overdiagnosis, or rare and complex skin conditions being missed. At a technology conference in the US on Tuesday, Google revealed there are almost 10bn Google searches related to skin, nail and hair issues every year. In response, Google has developed an artificial intelligence "dermatology assist tool" for people with concerns about their skin. Users of the app can use their phone to take three images of their skin, hair or nails from different angles. The app will then ask users questions about their skin type, how long they have had the issue, and for other symptoms that help narrow down the possibilities.

These days, it might seem like algorithms are out-diagnosing doctors at every turn, identifying dangerous lesions and dodgy moles with the unerring consistency only a machine can muster. Just this month, Google generated a wave of headlines with a study showing that its AI systems can spot breast cancer in mammograms more accurately than doctors. But for many in health care, what studies like these demonstrate is not just the promise of AI, but also its potential threat. They say that for all of the obvious abilities of algorithms to crunch data, the subtle, judgment-based skills of nurses and doctors are not so easily digitized. And in some areas where tech companies are pushing medical AI, this technology could exacerbate existing problems.

The use of artificial intelligence might increase the speed and the consistency of cancer diagnosis, but could also exacerbate the problem of overdiagnosis, according to a perspective article recently published in the New England Journal of Medicine by Adewole Adamson and Gilbert Welch, who suggest that this risk may be mitigated by overcoming the dichotomous classification between "cancer" and "not cancer". Supervised machine learning consists in the generation of decision-making algorithms starting from sets of images that pathologists have categorized as either "cancer" or "not cancer." "The computer system learns by judging its diagnosis against the external standard of pathological interpretation" Adewole Adamson, assistant professor of Internal Medicine at Dell Medical School at the University of Texas, explains. "Reliance on this external standard is problematic, however, since machine learning doesn't solve the central problem associated with cancer diagnosis: the lack of a histopathological gold standard." There is no single right answer to the question: "What constitutes cancer?"