Diagnosis Uncertain Models For Medical Risk Prediction

In-hospital patient outcome prediction is a major research area at the intersection of machine learning and medicine [Barfod et al., 2012, Taylor et al., 2016, Brajer et al., 2020, Naemi et al., 2021, Soffer et al., 2021, Wiesenfeld et al., 2022]. An important application of such models is'early' risk prediction - for example, using risk scores for triage [Raita et al., 2019, Klug et al., 2020]. Early prediction often requires calculating patient risk when primary diagnosis is still unknown or uncertain. We propose a method for incorporating uncertainty about diagnosis into mortality risk assessments in an interpretable and actionable way. We study the problem of all-cause in-hospital mortality prediction in the MIMIC-IV dataset [Johnson et al., 2023]. We find that a single model which pools all data and ignores diagnoses (we refer to this as the all-cause model or ACM) performs better at prediction than diagnosis-specific modeling. This increase in performance comes from the fact that the ACM has access to more data (so has lower variance) and that there is substantial transferrability in risk across diagnoses (so the ACM bias is not that high). We see this even more starkly by showing that a model trained only on out-of-diagnosis data can, due to this logic, predict risk within a diagnosis just as well as a model trained on that diagnosis only. While ACM are on average quite performant, we find that there are cases where they can fail.