In 2017, Google quietly published a blog post about a new approach to machine learning. Unlike the standard method, which requires the data to be centralized in one place, the new one could learn from a series of data sources distributed across multiple devices. The invention allowed Google to train its predictive text model on all the messages sent and received by Android users--without ever actually reading them or removing them from their phones. Despite its cleverness, federated learning, as the researchers called it, gained little traction within the AI community at the time. Now that is poised to change as it finds application in a completely new area: its privacy-first approach could very well be the answer to the greatest obstacle facing AI adoption in health care today.
Over the past decade, machine learning techniques have made substantial advances in many domains. In health care, global interest in the potential of machine learning has increased; for example, a deep learning algorithm has shown high accuracy in detecting diabetic retinopathy.1 There have been suggestions that machine learning will drive changes in health care within a few years, specifically in medical disciplines that require more accurate prognostic models (eg, oncology) and those based on pattern recognition (eg, radiology and pathology).
In 1976, Maxmen1 predicted that artificial intelligence (AI) in the 21st century would usher in "the post-physician era," with health care provided by paramedics and computers. Today, the mass extinction of physicians remains unlikely. However, as outlined by Hinton2 in a related Viewpoint, the emergence of a radically different approach to AI, called deep learning, has the potential to effect major changes in clinical medicine and health care delivery.
With public and academic attention increasingly focused on the new role of machine learning in the health information economy, an unusual and no-longer-esoteric category of vulnerabilities in machine-learning systems could prove important. These vulnerabilities allow a small, carefully designed change in how inputs are presented to a system to completely alter its output, causing it to confidently arrive at manifestly wrong conclusions. These advanced techniques to subvert otherwise-reliable machine-learning systems--so-called adversarial attacks--have, to date, been of interest primarily to computer science researchers (1). However, the landscape of often-competing interests within health care, and billions of dollars at stake in systems' outputs, implies considerable problems. We outline motivations that various players in the health care system may have to use adversarial attacks and begin a discussion of what to do about them.