Multimodal AI in Healthcare: Closing the Gaps

Healthcare professionals, in their daily routine, make use of multiple sources of data. To arrive to a diagnosis and decide on patient management, they rely on a combination of several types and sources of data: imaging (e.g., Radiology, Pathology, Ophthalmology), time series (e.g., electrocardiograms -- ECG), structured clinical data (e.g., vital signs, lab results) and non-structured data (e.g., clinical notes). Considering the level of expertise required to understand in depth one single data type, it is close to impossible for a single healthcare professional to master all areas. A radiologist has specialized training to read radiological images, but doesn't know as much about internal medicine or surgery. A cardiologist has a deep understanding of ECGs, but normally does not know how to evaluate a pathology slide.