A major challenge in reinforcement learning is to determine which state-action pairs are responsible for future rewards that are delayed.

Integrating multi-modal clinical data, such as electronic health records (EHR) and chest X-ray images (CXR), is particularly beneficial for clinical prediction tasks. However, in a temporal setting, multi-modal data are often inherently asynchronous.