Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer

F ondazione Policlinico Universitario Campus Bio-Medico, Rome, Italy Research Unit of Radiation Oncology, Department of Medicine and Surgery, Universit ` a Campus Bio-Medico di Roma, Rome, Italy Research Unit of Anatomical Pathology, Department of Medicine and Surgery, Universit ` a Campus Bio-Medico di Roma, Rome, Italy Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Ume a University, Ume a, Sweden Abstract --This study proposes a novel approach combining Multimodal Deep Learning with intrinsic eXplainable Artificial Intelligence techniques to predict pathological response in non-small cell lung cancer patients undergoing neoadjuvant therapy. Due to the limitations of existing radiomics and unimodal deep learning approaches, we introduce an intermediate fusion strategy that integrates imaging and clinical data, enabling efficient interaction between data modalities. The proposed Multimodal Doctor-in-the-Loop method further enhances clinical relevance by embedding clinicians' domain knowledge directly into the training process, guiding the model's focus gradually from broader lung regions to specific lesions. Results demonstrate improved predictive accuracy and explainability, providing insights into optimal data integration strategies for clinical applications. I NTRODUCTION Non-small cell lung cancer (NSCLC) is the most common subtype of lung cancer, constituting approximately 85% of lung cancer cases [1]. Currently, surgery remains the main treatment for early-stage and resectable locally advanced NSCLC, although a notable number of patients experience post-surgery recurrence. Neoadjuvant therapy (NA T) has shown potential in improving overall survival rates and reducing the risk of distant disease recurrence [2]. Achieving a complete pathological response after NA T, indicating the absence of tumor cells in all specimens, may have a potential prognostic role and serve as a surrogate survival endpoint [3]. Evaluating pathological response before surgical resection provides valuable insights into tumor sensitivity to the administered therapy, enabling clinicians to tailor the type of treatment to the needs of patients and reserve surgical interventions only for patients who are most likely to benefit from them.