dsc
Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts
Gordaliza, Pedro M., Molchanova, Nataliia, Banus, Jaume, Sanchez, Thomas, Cuadra, Meritxell Bach
Deep learning models for medical image segmentation suffer significant performance drops due to distribution shifts, but the causal mechanisms behind these drops remain poorly understood. We extend causal attribution frameworks to high-dimensional segmentation tasks, quantifying how acquisition protocols and annotation variability independently contribute to performance degradation. We model the data-generating process through a causal graph and employ Shapley values to fairly attribute performance changes to individual mechanisms. Our framework addresses unique challenges in medical imaging: high-dimensional outputs, limited samples, and complex mechanism interactions. Validation on multiple sclerosis (MS) lesion segmentation across 4 centers and 7 annotators reveals context-dependent failure modes: annotation protocol shifts dominate when crossing annotators (7.4% $\pm$ 8.9% DSC attribution), while acquisition shifts dominate when crossing imaging centers (6.5% $\pm$ 9.1%). This mechanism-specific quantification enables practitioners to prioritize targeted interventions based on deployment context.
TAP-CT: 3D Task-Agnostic Pretraining of Computed Tomography Foundation Models
Veenboer, Tim, Yiasemis, George, Marcus, Eric, Van Veldhuizen, Vivien, Snoek, Cees G. M., Teuwen, Jonas, Lipman, Kevin B. W. Groot
Existing foundation models (FMs) in the medical domain often require extensive fine-tuning or rely on training resource-intensive decoders, while many existing encoders are pretrained with objectives biased toward specific tasks. This illustrates a need for a strong, task-agnostic foundation model that requires minimal fine-tuning beyond feature extraction. In this work, we introduce a suite of task-agnostic pretraining of CT foundation models (TAP-CT): a simple yet effective adaptation of Vision Transformers (ViTs) and DINOv2 for volumetric data, enabling scalable self-supervised pretraining directly on 3D CT volumes. Our approach incorporates targeted modifications to patch embeddings, positional encodings, and volumetric augmentations, making the architecture depth-aware while preserving the simplicity of the underlying architectures. We show that large-scale 3D pretraining on an extensive in-house CT dataset (105K volumes) yields stable, robust frozen representations that generalize strongly across downstream tasks. To promote transparency and reproducibility, and to establish a powerful, low-resource baseline for future research in medical imaging, we will release all pretrained models, experimental configurations, and downstream benchmark code at https://huggingface.co/fomofo/tap-ct-b-3d.
NeuroVascU-Net: A Unified Multi-Scale and Cross-Domain Adaptive Feature Fusion U-Net for Precise 3D Segmentation of Brain Vessels in Contrast-Enhanced T1 MRI
Vayeghan, Mohammad Jafari, Delfan, Niloufar, Masouleh, Mehdi Tale, Rizi, Mansour Parvaresh, Moshiri, Behzad
Precise 3D segmentation of cerebral vasculature from T1-weighted contrast-enhanced (T1CE) MRI is crucial for safe neurosurgical planning. Manual delineation is time-consuming and prone to inter-observer variability, while current automated methods often trade accuracy for computational cost, limiting clinical use. We present NeuroVascU-Net, the first deep learning architecture specifically designed to segment cerebrovascular structures directly from clinically standard T1CE MRI in neuro-oncology patients, addressing a gap in prior work dominated by TOF-MRA-based approaches. NeuroVascU-Net builds on a dilated U-Net and integrates two specialized modules: a Multi-Scale Contextual Feature Fusion ($MSC^2F$) module at the bottleneck and a Cross-Domain Adaptive Feature Fusion ($CDA^2F$) module at deeper hierarchical layers. $MSC^2F$ captures both local and global information via multi-scale dilated convolutions, while $CDA^2F$ dynamically integrates domain-specific features, enhancing representation while keeping computation low. The model was trained and validated on a curated dataset of T1CE scans from 137 brain tumor biopsy patients, annotated by a board-certified functional neurosurgeon. NeuroVascU-Net achieved a Dice score of 0.8609 and precision of 0.8841, accurately segmenting both major and fine vascular structures. Notably, it requires only 12.4M parameters, significantly fewer than transformer-based models such as Swin U-NetR. This balance of accuracy and efficiency positions NeuroVascU-Net as a practical solution for computer-assisted neurosurgical planning.
Accelerating Volumetric Medical Image Annotation via Short-Long Memory SAM 2
Chen, Yuwen, Yildiz, Zafer, Li, Qihang, Chen, Yaqian, Dong, Haoyu, Gu, Hanxue, Konz, Nicholas, Mazurowski, Maciej A.
Manual annotation of volumetric medical images, such as magnetic resonance imaging (MRI) and computed tomography (CT), is a labor-intensive and time-consuming process. Recent advancements in foundation models for video object segmentation, such as Segment Anything Model 2 (SAM 2), offer a potential opportunity to significantly speed up the annotation process by manually annotating one or a few slices and then propagating target masks across the entire volume. However, the performance of SAM 2 in this context varies. Our experiments show that relying on a single memory bank and attention module is prone to error propagation, particularly at boundary regions where the target is present in the previous slice but absent in the current one. To address this problem, we propose Short-Long Memory SAM 2 (SLM-SAM 2), a novel architecture that integrates distinct short-term and long-term memory banks with separate attention modules to improve segmentation accuracy. We evaluate SLM-SAM 2 on four public datasets covering organs, bones, and muscles across MRI, CT, and ultrasound videos. We show that the proposed method markedly outperforms the default SAM 2, achieving an average Dice Similarity Coefficient improvement of 0.14 and 0.10 in the scenarios when 5 volumes and 1 volume are available for the initial adaptation, respectively. SLM-SAM 2 also exhibits stronger resistance to over-propagation, reducing the time required to correct propagated masks by 60.575% per volume compared to SAM 2, making a notable step toward more accurate automated annotation of medical images for segmentation model development.
Generalisation of automatic tumour segmentation in histopathological whole-slide images across multiple cancer types
Skrede, Ole-Johan, Pradhan, Manohar, Isaksen, Maria Xepapadakis, Hveem, Tarjei Sveinsgjerd, Vlatkovic, Ljiljana, Nesbakken, Arild, Lindemann, Kristina, Kristensen, Gunnar B, Kasius, Jenneke, Zeimet, Alain G, Brustugun, Odd Terje, Busund, Lill-Tove Rasmussen, Richardsen, Elin H, Haug, Erik Skaaheim, Brennhovd, Bjรธrn, Rewcastle, Emma, Lillesand, Melinda, Kvikstad, Vebjรธrn, Janssen, Emiel, Kerr, David J, Liestรธl, Knut, Albregtsen, Fritz, Kleppe, Andreas
Deep learning is expected to aid pathologists by automating tasks such as tumour segmentation. We aimed to develop one universal tumour segmentation model for histopathological images and examine its performance in different cancer types. The model was developed using over 20 000 whole-slide images from over 4 000 patients with colorectal, endometrial, lung, or prostate carcinoma. Performance was validated in pre-planned analyses on external cohorts with over 3 000 patients across six cancer types. Exploratory analyses included over 1 500 additional patients from The Cancer Genome Atlas. Average Dice coefficient was over 80% in all validation cohorts with en bloc resection specimens and in The Cancer Genome Atlas cohorts. No loss of performance was observed when comparing the universal model with models specialised on single cancer types. In conclusion, extensive and rigorous evaluations demonstrate that generic tumour segmentation by a single model is possible across cancer types, patient populations, sample preparations, and slide scanners.
Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Models
Rasaee, Hamza, Koleilat, Taha, Rivaz, Hassan
Abstract-- Accurate and generalizable object segmentation in ultrasound imaging remains a significant challenge due to anatomical variability, diverse imaging protocols, and limited annotated data. In this study, we propose a prompt-driven vision-language model (VLM) that integrates Grounding DINO with SAM2 (Segment Anything Model2) to enable object segmentation across multiple ultrasound organs. A total of 18 public ultrasound datasets, encompassing the breast, thyroid, liver, prostate, kidney, and paraspinal muscle, were utilized. These datasets were divided into 15 for fine-tuning and validation of Grounding DINO using Low Rank Adaptation (LoRA) to the ultrasound domain, and 3 were held out entirely for testing to evaluate performance in unseen distributions. Comprehensive experiments demonstrate that our approach outperforms state-of-the-art segmentation methods, including UniverSeg, MedSAM, MedCLIP-SAM, BiomedParse, and SAMUS on most seen datasets while maintaining strong performance on unseen datasets without additional fine-tuning. These results underscore the promise of VLMs in scalable and robust ultrasound image analysis, reducing dependence on large, organ-specific annotated datasets. We will publish our code on code. Ultrasound imaging is extensively used in clinical practice due to its safety, affordability, portability, and real-time capabilities. It plays a vital role in cancer screening, disease staging, and image-guided interventions across various anatomies, including the breast, thyroid, liver, prostate, kidney, and musculoskeletal system. Despite these advantages, ultrasound imaging presents intrinsic challenges that complicate automated analysis. Issues like low tissue contrast, speckle noise, acoustic shadowing, and operator-dependent variability degrade image quality and hinder the precise delineation of anatomical structures, ultimately affecting automated segmentation algorithms' performance and generalizability.
Multimodal Contrastive Pretraining of CBCT and IOS for Enhanced Tooth Segmentation
Son, Moo Hyun, Bae, Juyoung, Qiu, Zelin, Peng, Jiale, Li, Kai Xin, Lin, Yifan, Chen, Hao
Oral diseases remain one of the most pervasive global health issues, affecting over 3.5 billion individuals, which accounts for over 43% of the global population as reported by the World Health Organization [1]. This widespread prevalence underscores the critical importance of dentistry, not only for clinical needs but also for enhancing the overall quality of life for a large portion of the global population. In modern dental practice, digital dentistry plays a crucial role in streamlining workflows and enhancing patient outcomes. Cone-Beam Computed Tomography (CBCT) visualizes 3D anatomical structures, including tooth morphology, alveolar bone, and surrounding tissues [2], while intraoral scans (IOS) provide high-resolution images of occlusal surfaces that are crucial for treatment planning and prosthesis design [3]. However, these imaging modalities still require extensive manual and time-consuming analysis to identify and plan treatments [4]. Consequently, numerous research efforts now focus on automating key tasks such as caries detection [5-7], orthodontic treatment planning [8-10], and designing dental prostheses, including implants, crowns, and bridges [11-13].
Stress-testing cross-cancer generalizability of 3D nnU-Net for PET-CT tumor segmentation: multi-cohort evaluation with novel oesophageal and lung cancer datasets
Ghosh, Soumen, Hannan, Christine Jestin, Vashistha, Rajat, Kundu, Parveen, Brosda, Sandra, Aoude, Lauren G., Lonie, James, Nathanson, Andrew, Ng, Jessica, Barbour, Andrew P., Vegh, Viktor
Robust generalization is essential for deploying deep learning based tumor segmentation in clinical PET-CT workflows, where anatomical sites, scanners, and patient populations vary widely. This study presents the first cross cancer evaluation of nnU-Net on PET-CT, introducing two novel, expert-annotated whole-body datasets. 279 patients with oesophageal cancer (Australian cohort) and 54 with lung cancer (Indian cohort). These cohorts complement the public AutoPET dataset and enable systematic stress-testing of cross domain performance. We trained and tested 3D nnUNet models under three paradigms. Target only (oesophageal), public only (AutoPET), and combined training. For the tested sets, the oesophageal only model achieved the best in-domain accuracy (mean DSC, 57.8) but failed on external Indian lung cohort (mean DSC less than 3.4), indicating severe overfitting. The public only model generalized more broadly (mean DSC, 63.5 on AutoPET, 51.6 on Indian lung cohort) but underperformed in oesophageal Australian cohort (mean DSC, 26.7). The combined approach provided the most balanced results (mean DSC, lung (52.9), oesophageal (40.7), AutoPET (60.9)), reducing boundary errors and improving robustness across all cohorts. These findings demonstrate that dataset diversity, particularly multi demographic, multi center and multi cancer integration, outweighs architectural novelty as the key driver of robust generalization. This work presents the demography based cross cancer deep learning segmentation evaluation and highlights dataset diversity, rather than model complexity, as the foundation for clinically robust segmentation.