A Comparison of Object Detection and Phrase Grounding Models in Chest X-ray Abnormality Localization using Eye-tracking Data

ABSTRACT Chest diseases rank among the most prevalent and dangerous global health issues. Object detection and phrase groundin g deep learning models interpret complex radiology data to as - sist healthcare professionals in diagnosis. Object detect ion locates abnormalities for classes, while phrase grounding locates abnormalities for textual descriptions. This paper i nves-tigates how text enhances abnormality localization in ches t X-rays by comparing the performance and explainability of these two tasks. To establish an explainability benchmark, we proposed an automatic pipeline to generate image regions for report sentences using radiologists' eye-tracking dat a Index T erms -- Multi-Modal Learning, Localization, Eye-tracking Data, Data Generation, XAI 1. INTRODUCTION Since the emergence of deep neural networks (DNN), they have been applied to various medical domains and applications.