Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling

Purpose: Interpreting chest radiographs (CXR) remains challenging due to the ambiguity of overlapping structures such as the lungs, heart, and bones. To address this issue, we propose a novel method for extracting fine-grained anatomical structures in CXR using pseudo-labeling of three-dimensional computed tomography (CT) scans. Methods: We created a large-scale dataset of 10,021 thoracic CTs with 157 labels and applied an ensemble of 3D anatomy segmentation models to extract anatomical pseudo-labels. These labels were projected onto a two-dimensional plane, similar to the CXR, allowing the training of detailed semantic segmentation models for CXR without any manual annotation effort. Our anatomical segmentations allowed for the accurate extraction of relevant explainable medical features such as the cardio-thoracic-ratio. Conclusion: Our method of volumetric pseudo-labeling paired with CT projection offers a promising approach for detailed anatomical segmentation of CXR with a high agreement with human annotators. This technique may have important clinical implications, particularly in the analysis of various thoracic pathologies. Chest radiographs (CXR) are one of the most common diagnostic imaging methods for patients with respiratory or cardiovascular conditions, with more than 130 million studies performed annually in Germany alone [1]. By using ionizing radiation to penetrate the body, CXR provide a visual representation of the organs, tissues and cavities and their current state. The interpretation of CXR these images is challenging, since it requires a thorough understanding of human anatomy due to the presence of overlapping structures that can obscure potential pathological changes and other abnormalities. Despite these challenges, CXR remain a standard diagnostic procedure and its quantitative analysis can be time consuming. With the increasing demand in imaging procedures and the massive workload that comes along with it, this can lead to avoidable errors due to rushed examination [2, 3] or burnout due to the overly straining of doctors [4-6]. Recent advances in Computer Vision, such as convolutional neural networks (CNN) or vision transformers (ViT), have the potential to reduce the workload of radiologists in both image analysis and reporting [7, 8].