Multimodal AI for medicine, simplified - by Eric Topol

AI in medicine is basically a single mode story to date. Help reading an X-ray or MRI, finding polyps during a colonoscopy, provide patient coaching for a specific condition like diabetes, or a preliminary diagnosis of a skin lesion or heart rhythm from a smartwatch recording. It has largely been image-centric to date, with minimal integration with, or use of inputs from, text and voice. But over time that narrowness and constraint may well be alleviated. My colleagues and I recently wrote a review of the potential for multimodal AI, when data from many sources can be ingested and processed as seen below.