Estimation of patient acuity in the Intensive Care Unit (ICU) is vital to ensure timely and appropriate interventions. Advances in artificial intelligence (AI) technologies have significantly improved the accuracy of acuity predictions. However, prior studies using machine learning for acuity prediction have predominantly relied on electronic health records (EHR) data, often overlooking other critical aspects of ICU stay, such as patient mobility, environmental factors, and facial cues indicating pain or agitation. To address this gap, we present MANGO: the Multimodal Acuity traNsformer for intelliGent ICU Outcomes, designed to enhance the prediction of patient acuity states, transitions, and the need for life-sustaining therapy. We collected a multimodal dataset ICU-Multimodal, incorporating four key modalities: EHR data, wearable sensor data, video of patient's facial cues, and ambient sensor data, which we utilized to train MANGO. The MANGO model employs a multimodal feature fusion network powered by Transformer masked self-attention method, enabling it to capture and learn complex interactions across these diverse data modalities even when some modalities are absent. Our results demonstrated that integrating multiple modalities significantly improved the model's ability to predict acuity status, transitions, and the need for life-sustaining therapy. The best-performing models achieved an area under the receiver operating characteristic curve (AUROC) of 0.76 (95% CI: 0.72-0.79)

Background: In the United States, 5.7 million patients are admitted annually to intensive care units (ICU), with costs exceeding $82 billion. Although close monitoring and dynamic assessment of patient acuity are key aspects of ICU care, both are limited by the time constraints imposed on healthcare providers. Methods: Using the University of Florida Health (UFH) Integrated Data Repository as Honest Broker, we created a database with electronic health records data from a retrospective study cohort of 38,749 adult patients admitted to ICU at UF Health between 06/01/2014 and 08/22/2019. This repository includes demographic information, comorbidities, vital signs, laboratory values, medications with date and timestamps, and diagnoses and procedure codes for all index admission encounters as well as encounters within 12 months prior to index admission and 12 months follow-up. We developed algorithms to identify acuity status of the patient every four hours during each ICU stay. Results: We had 383,193 encounters (121,800 unique patients) admitted to the hospital, and 51,073 encounters (38,749 unique patients) with at least one ICU stay that lasted more than four hours. These patients requiring ICU admission had longer median hospital stay (7 days vs. 1 day) and higher in-hospital mortality (9.6% vs. 0.4%) compared with those not admitted to the ICU. Among patients who were admitted to the ICU and expired during hospital admission, more deaths occurred in the ICU than on general hospital wards (7.4% vs. 0.8%, respectively). Conclusions: We developed phenotyping algorithms that determined patient acuity status every four hours while admitted to the ICU. This approach may be useful in developing prognostic and clinical decision-support tools to aid patients, caregivers, and providers in shared decision-making processes regarding resource use and escalation of care.