Goto

Collaborating Authors

 pathologist




Autonomous labeling of surgical resection margins using a foundation model

Yang, Xilin, Aydin, Musa, Lu, Yuhong, Selcuk, Sahan Yoruc, Bai, Bijie, Zhang, Yijie, Birkeland, Andrew, Ehrlich, Katjana, Bec, Julien, Marcu, Laura, Pillar, Nir, Ozcan, Aydogan

arXiv.org Artificial Intelligence

Assessing resection margins is central to pathological specimen evaluation and has profound implications for patient outcomes. Current practice employs physical inking, which is applied variably, and cautery artifacts can obscure the true margin on histological sections. We present a virtual inking network (VIN) that autonomously localizes the surgical cut surface on whole-slide images, reducing reliance on inks and standardizing margin-focused review. VIN uses a frozen foundation model as the feature extractor and a compact two-layer multilayer perceptron trained for patch-level classification of cautery-consistent features. The dataset comprised 120 hematoxylin and eosin (H&E) stained slides from 12 human tonsil tissue blocks, resulting in ~2 TB of uncompressed raw image data, where a board-certified pathologist provided boundary annotations. In blind testing with 20 slides from previously unseen blocks, VIN produced coherent margin overlays that qualitatively aligned with expert annotations across serial sections. Quantitatively, region-level accuracy was ~73.3% across the test set, with errors largely confined to limited areas that did not disrupt continuity of the whole-slide margin map. These results indicate that VIN captures cautery-related histomorphology and can provide a reproducible, ink-free margin delineation suitable for integration into routine digital pathology workflows and for downstream measurement of margin distances.


HistoLens: An Interactive XAI Toolkit for Verifying and Mitigating Flaws in Vision-Language Models for Histopathology

Vissapragada, Sandeep, Sahu, Vikrant, Gupta, Gagan Raj, Singh, Vandita

arXiv.org Artificial Intelligence

For doctors to truly trust artificial intelligence, it can't be a black box. They need to understand its reasoning, almost as if they were consulting a colleague. We created HistoLens1 to be that transparent, collaborative partner. It allows a pathologist to simply ask a question in plain English about a tissue slide--just as they would ask a trainee. Our system intelligently translates this question into a precise query for its AI engine, which then provides a clear, structured report. But it doesn't stop there. If a doctor ever asks, "Why?", HistoLens can instantly provide a 'visual proof' for any finding--a heatmap that points to the exact cells and regions the AI used for its analysis. We've also ensured the AI focuses only on the patient's tissue, just like a trained pathologist would, by teaching it to ignore distracting background noise. The result is a workflow where the pathologist remains the expert in charge, using a trustworthy AI assistant to verify their insights and make faster, more confident diagnoses.


MSDM: Generating Task-Specific Pathology Images with a Multimodal Conditioned Diffusion Model for Cell and Nuclei Segmentation

Winter, Dominik, Bui, Mai, Gavaldon, Monica Azqueta, Triltsch, Nicolas, Rosati, Marco, Brieu, Nicolas

arXiv.org Artificial Intelligence

Scarcity of annotated data, particularly for rare or atypical morphologies, present significant challenges for cell and nuclei segmentation in computational pathology. While manual annotation is labor-intensive and costly, synthetic data offers a cost-effective alternative. We introduce a Multimodal Semantic Diffusion Model (MSDM) for generating realistic pixel-precise image-mask pairs for cell and nuclei segmentation. By conditioning the generative process with cellular/nuclear morphologies (using horizontal and vertical maps), RGB color characteristics, and BERT-encoded assay/indication metadata, MSDM generates datasests with desired morphological properties. These heterogeneous modalities are integrated via multi-head cross-attention, enabling fine-grained control over the generated images. Quantitative analysis demonstrates that synthetic images closely match real data, with low Wasserstein distances between embeddings of generated and real images under matching biological conditions. The incorporation of these synthetic samples, exemplified by columnar cells, significantly improves segmentation model accuracy on columnar cells. This strategy systematically enriches data sets, directly targeting model deficiencies. We highlight the effectiveness of multimodal diffusion-based augmentation for advancing the robustness and generalizability of cell and nuclei segmentation models. Thereby, we pave the way for broader application of generative models in computational pathology.


Finding Holes: Pathologist Level Performance Using AI for Cribriform Morphology Detection in Prostate Cancer

Szolnoky, Kelvin, Blilie, Anders, Mulliqi, Nita, Tsuzuki, Toyonori, Samaratunga, Hemamali, Titus, Matteo, Ji, Xiaoyi, Boman, Sol Erika, Gudlaugsson, Einar, Kjosavik, Svein Reidar, Asenjo, José, Gambacorta, Marcello, Libretti, Paolo, Braun, Marcin, Kordek, Radisław, Łowicki, Roman, Delahunt, Brett, Iczkowski, Kenneth A., van der Kwast, Theo, van Leenders, Geert J. L. H., Leite, Katia R. M., Pan, Chin-Chen, Janssen, Emiel Adrianus Maria, Eklund, Martin, Egevad, Lars, Kartasalo, Kimmo

arXiv.org Artificial Intelligence

Background: Cribriform morphology in prostate cancer is a histological feature that indicates poor prognosis and contraindicates active surveillance. However, it remains underreported and subject to significant interobserver variability amongst pathologists. We aimed to develop and validate an AI-based system to improve cribriform pattern detection. Methods: We created a deep learning model using an EfficientNetV2-S encoder with multiple instance learning for end-to-end whole-slide classification. The model was trained on 640 digitised prostate core needle biopsies from 430 patients, collected across three cohorts. It was validated internally (261 slides from 171 patients) and externally (266 slides, 104 patients from three independent cohorts). Internal validation cohorts included laboratories or scanners from the development set, while external cohorts used completely independent instruments and laboratories. Annotations were provided by three expert uropathologists with known high concordance. Additionally, we conducted an inter-rater analysis and compared the model's performance against nine expert uropathologists on 88 slides from the internal validation cohort. Results: The model showed strong internal validation performance (AUC: 0.97, 95% CI: 0.95-0.99; Cohen's kappa: 0.81, 95% CI: 0.72-0.89) and robust external validation (AUC: 0.90, 95% CI: 0.86-0.93; Cohen's kappa: 0.55, 95% CI: 0.45-0.64). In our inter-rater analysis, the model achieved the highest average agreement (Cohen's kappa: 0.66, 95% CI: 0.57-0.74), outperforming all nine pathologists whose Cohen's kappas ranged from 0.35 to 0.62. Conclusion: Our AI model demonstrates pathologist-level performance for cribriform morphology detection in prostate cancer. This approach could enhance diagnostic reliability, standardise reporting, and improve treatment decisions for prostate cancer patients.


SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space

Redekop, Ekaterina, Pleasure, Mara, Wang, Zichen, Flores, Kimberly, Sisk, Anthony, Speier, William, Arnold, Corey W.

arXiv.org Artificial Intelligence

The rapid growth of digital pathology and advances in self-supervised deep learning have enabled the development of foundational models for various pathology tasks across diverse diseases. While multimodal approaches integrating diverse data sources have emerged, a critical gap remains in the comprehensive integration of whole-slide images (WSIs) with spatial tran-scriptomics (ST), which is crucial for capturing critical molecular heterogeneity beyond standard hematoxylin & eosin (H&E) staining. We introduce SPADE, a foundation model that integrates histopathology with ST data to guide image representation learning within a unified framework, in effect creating an ST-informed latent space. These authors contributed equally to this work. Pre-trained on the comprehensive HEST-1k dataset, SPADE is evaluated on 20 downstream tasks, demonstrating significantly superior few-shot performance compared to baseline models, highlighting the benefits of integrating morphological and molecular information into one latent space. Introduction High-resolution whole slide images (WSIs) have propelled the development of powerful deep learning foundation models in computational pathology, demonstrating robust performance across diverse tissue types and tasks [1, 2, 3, 4]. These models are typically trained using self-supervision, enabling learning from large unlabeled datasets and producing embeddings robust to institutional variations, including differences in staining procedures and other image-quality factors [5, 6, 7, 8]. By visually capturing cellular arrangement, WSIs enable the study of spatial organization and disorganization of cells in tissues, characterizations that are especially relevant in cancer research [9, 10]. In clinical settings, WSIs are commonly stained with hematoxylin & eosin (H&E), a two-color stain that highlights nuclei and cytoplasm but offers a limited view of molecular-level heterogeneity [11]. As tumor tissues are known to exhibit high variability within and across patients, deciphering the heterogeneity at the molecular level is critical for improving deep learning applications that can more precisely inform diagnosis, treatment, and prognosis [12, 13]. While H&E provides crucial morphological insights, its inability to capture molecular heterogeneity limits its utility in fully characterizing tissue complexity. Spatial transcriptomics addresses this gap by providing spatially resolved gene expression data, allowing for additional molecular context for a given tissue specimen. Although both ST and H&E data have independently proven useful in various applications, their combined potential for creating a more comprehensive representation learning framework remains unexplored. To this end, we introduce SPADE, a vision-ST foundation model that uses a mixture of experts, each trained via contrastive learning, to unify ST data and H&E images to produce slide representations that encompass both modalities.


Generalisation of automatic tumour segmentation in histopathological whole-slide images across multiple cancer types

Skrede, Ole-Johan, Pradhan, Manohar, Isaksen, Maria Xepapadakis, Hveem, Tarjei Sveinsgjerd, Vlatkovic, Ljiljana, Nesbakken, Arild, Lindemann, Kristina, Kristensen, Gunnar B, Kasius, Jenneke, Zeimet, Alain G, Brustugun, Odd Terje, Busund, Lill-Tove Rasmussen, Richardsen, Elin H, Haug, Erik Skaaheim, Brennhovd, Bjørn, Rewcastle, Emma, Lillesand, Melinda, Kvikstad, Vebjørn, Janssen, Emiel, Kerr, David J, Liestøl, Knut, Albregtsen, Fritz, Kleppe, Andreas

arXiv.org Artificial Intelligence

Deep learning is expected to aid pathologists by automating tasks such as tumour segmentation. We aimed to develop one universal tumour segmentation model for histopathological images and examine its performance in different cancer types. The model was developed using over 20 000 whole-slide images from over 4 000 patients with colorectal, endometrial, lung, or prostate carcinoma. Performance was validated in pre-planned analyses on external cohorts with over 3 000 patients across six cancer types. Exploratory analyses included over 1 500 additional patients from The Cancer Genome Atlas. Average Dice coefficient was over 80% in all validation cohorts with en bloc resection specimens and in The Cancer Genome Atlas cohorts. No loss of performance was observed when comparing the universal model with models specialised on single cancer types. In conclusion, extensive and rigorous evaluations demonstrate that generic tumour segmentation by a single model is possible across cancer types, patient populations, sample preparations, and slide scanners.


A Histological

Neural Information Processing Systems

These images were evenly split between cases diagnosed with adenocarcinoma of the lung and squamous cell carcinoma, representing the two most common sub-types in lung cancer. The images were scanned on an Aperio scanner at a resolution of 0 . Different classes used for conditioning were annotated digitally by a pathologist using an apple pencil with the instruction to clearly demarcate boundaries between tissue regions. The pathologist could choose from a list of 40 distinct annotation categories, aiming to cover all possible annotation requirements. All data handling was performed in strict accordance with privacy regulations and ethical standards, ensuring the protection of patient information at all times.


DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology

Neural Information Processing Systems

We present DiffInfinite, a hierarchical diffusion model that generates arbitrarily large histological images while preserving long-range correlation structural information. Our approach first generates synthetic segmentation masks, subsequently used as conditions for the high-fidelity generative diffusion process.