DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models

Referring remote sensing image segmentation (RRSIS) enables the precise delineation of regions within remote sensing imagery through natural language descriptions, serving critical applications in disaster response, urban development, and environmental monitoring. Despite recent advances, current approaches face significant challenges in processing aerial imagery due to complex object characteristics including scale variations, diverse orientations, and semantic ambiguities inherent to the overhead perspective. To address these limitations, we propose Di ffRIS, a novel framework that harnesses the semantic understanding capabilities of pre-trained text-to-image di ff usion models for enhanced cross-modal alignment in RRSIS tasks. Our framework introduces two key innovations: a context perception adapter (CP-adapter) that dynamically refines linguistic features through global context modeling and object-aware reasoning, and a progressive cross-modal reasoning decoder (PCMRD) that iteratively aligns textual descriptions with visual regions for precise segmentation. The CP-adapter bridges the domain gap between general vision-language understanding and remote sensing applications, while PCMRD enables fine-grained semantic alignment through multi-scale feature interaction. Comprehensive experiments on three benchmark datasets--RRSIS-D, RefSegRS, and RISBench--demonstrate that Di ffRIS consistently outperforms existing methods across all standard metrics, establishing a new state-of-the-art for RRSIS tasks. The significant performance improvements validate the e ff ectiveness of leveraging pre-trained di ff usion models for remote sensing applications through our proposed adaptive framework. Introduction Referring remote sensing image segmentation (RRSIS) aims to identify specific regions in remote sensing imagery based on given textual conditions, making it particularly suitable for practical applications such as defense reconnaissance[1], climate impact studies[2], urban infrastructure management[3], and land use categorization[4]. Unlike traditional single-modal segmentation methods[5, 6], RRSIS leverages textual descriptions to guide image segmentation, overcoming the limitations of fixed category labels and enabling the processing of more diverse vocabulary and syntactic variations.