Weakly-Supervised Multimodal Learning on MIMIC-CXR

Agostini, Andrea, Chopard, Daphné, Meng, Yang, Fortin, Norbert, Shahbaba, Babak, Mandt, Stephan, Sutter, Thomas M., Vogt, Julia E.

arXiv.org Artificial Intelligence 

Multimodal data integration and label scarcity pose significant challenges for machine learning in medical settings. To address these issues, we conduct an in-depth evaluation of the newly proposed Multimodal Variational Mixture-of-Experts (MMVM) VAE on the challenging MIMIC-CXR dataset. Our analysis demonstrates that the MMVM VAE consistently outperforms other multimodal VAEs and fully supervised approaches, highlighting its strong potential for real-world medical applications.