Getting a quick and accurate reading of an X-ray or some other medical images can be vital to a patient's health and might even save a life. Obtaining such an assessment depends on the availability of a skilled radiologist and, consequently, a rapid response is not always possible. For that reason, says Ruizhi "Ray" Liao, a postdoc and a recent PhD graduate at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), "we want to train machines that are capable of reproducing what radiologists do every day." Liao is first author of a new paper, written with other researchers at MIT and Boston-area hospitals, that is being presented this fall at MICCAI 2021, an international conference on medical image computing. Although the idea of utilizing computers to interpret images is not new, the MIT-led group is drawing on an underused resource--the vast body of radiology reports that accompany medical images, written by radiologists in routine clinical practice--to improve the interpretive abilities of machine learning algorithms.

Getting a quick and accurate reading of an X-ray or some other medical images can be vital to a patient's health and might even save a life. Obtaining such an assessment depends on the availability of a skilled radiologist and, consequently, a rapid response is not always possible. For that reason, says Ruizhi "Ray" Liao, a postdoc and a recent PhD graduate at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), "we want to train machines that are capable of reproducing what radiologists do every day." Liao is first author of a new paper, written with other researchers at MIT and Boston-area hospitals, that is being presented this fall at MICCAI 2021, an international conference on medical image computing. Although the idea of utilizing computers to interpret images is not new, the MIT-led group is drawing on an underused resource -- the vast body of radiology reports that accompany medical images, written by radiologists in routine clinical practice -- to improve the interpretive abilities of machine learning algorithms.

The placenta is one of the most vital organs when a woman is pregnant. If it's not working correctly, the consequences can be dire: Children may experience stunted growth and neurological disorders, and their mothers are at increased risk of blood conditions like preeclampsia, which can impair kidney and liver function. Unfortunately, assessing placental health is difficult because of the limited information that can be gleaned from imaging. Traditional ultrasounds are cheap, portable, and easy to perform, but they can't always capture enough detail. This has spurred researchers to explore the potential of magnetic resonance imaging (MRI).

People who suffer a stroke often undergo a brain scan at the hospital, allowing doctors to determine the location and extent of the damage. Researchers who study the effects of strokes would love to be able to analyze these images, but the resolution is often too low for many analyses. To help scientists take advantage of this untapped wealth of data from hospital scans, a team of MIT researchers, working with doctors at Massachusetts General Hospital and many other institutions, has devised a way to boost the quality of these scans so they can be used for large-scale studies of how strokes affect different people and how they respond to treatment. "These images are quite unique because they are acquired in routine clinical practice when a patient comes in with a stroke," says Polina Golland, an MIT professor of electrical engineering and computer science. Using these scans, researchers could study how genetic factors influence stroke survival or how people respond to different treatments.

MIT researchers are developing a computer system that uses genetic, demographic, and clinical data to help predict the effects of disease on brain anatomy. In experiments, they trained a machine-learning system on MRI data from patients with neurodegenerative diseases and found that supplementing that training with other patient information improved the system's predictions. "This is the first paper that we've ever written on this," says Polina Golland, a professor of electrical engineering and computer science at MIT and the senior author on the new paper. "Our goal is not to prove that our model is the best model to do this kind of thing; it's to prove that the information is actually in the data. So what we've done is, we take our model, and we turn off the genetic information and the demographic and clinical information, and we see that with combined information, we can predict anatomical changes better."