deepmerge
DeepMerge: Deep-Learning-Based Region-Merging for Image Segmentation
Lv, Xianwei, Persello, Claudio, Li, Wangbin, Huang, Xiao, Ming, Dongping, Stein, Alfred
Image segmentation aims to partition an image according to the objects in the scene and is a fundamental step in analysing very high spatial-resolution (VHR) remote sensing imagery. Current methods struggle to effectively consider land objects with diverse shapes and sizes. Additionally, the determination of segmentation scale parameters frequently adheres to a static and empirical doctrine, posing limitations on the segmentation of large-scale remote sensing images and yielding algorithms with limited interpretability. To address the above challenges, we propose a deep-learning-based region merging method dubbed DeepMerge to handle the segmentation of complete objects in large VHR images by integrating deep learning and region adjacency graph (RAG). This is the first method to use deep learning to learn the similarity and merge similar adjacent super-pixels in RAG. We propose a modified binary tree sampling method to generate shift-scale data, serving as inputs for transformer-based deep learning networks, a shift-scale attention with 3-Dimension relative position embedding to learn features across scales, and an embedding to fuse learned features with hand-crafted features. DeepMerge can achieve high segmentation accuracy in a supervised manner from large-scale remotely sensed images and provides an interpretable optimal scale parameter, which is validated using a remote sensing image of 0.55 m resolution covering an area of 5,660 km^2. The experimental results show that DeepMerge achieves the highest F value (0.9550) and the lowest total error TE (0.0895), correctly segmenting objects of different sizes and outperforming all competing segmentation methods.
DeepMerge II: Building Robust Deep Learning Algorithms for Merging Galaxy Identification Across Domains
Ćiprijanović, A., Kafkes, D., Downey, K., Jenkins, S., Perdue, G. N., Madireddy, S., Johnston, T., Snyder, G. F., Nord, B.
In astronomy, neural networks are often trained on simulation data with the prospect of being used on telescope observations. Unfortunately, training a model on simulation data and then applying it to instrument data leads to a substantial and potentially even detrimental decrease in model accuracy on the new target dataset. Simulated and instrument data represent different data domains, and for an algorithm to work in both, domain-invariant learning is necessary. Here we employ domain adaptation techniques$-$ Maximum Mean Discrepancy (MMD) as an additional transfer loss and Domain Adversarial Neural Networks (DANNs)$-$ and demonstrate their viability to extract domain-invariant features within the astronomical context of classifying merging and non-merging galaxies. Additionally, we explore the use of Fisher loss and entropy minimization to enforce better in-domain class discriminability. We show that the addition of each domain adaptation technique improves the performance of a classifier when compared to conventional deep learning algorithms. We demonstrate this on two examples: between two Illustris-1 simulated datasets of distant merging galaxies, and between Illustris-1 simulated data of nearby merging galaxies and observed data from the Sloan Digital Sky Survey. The use of domain adaptation techniques in our experiments leads to an increase of target domain classification accuracy of up to ${\sim}20\%$. With further development, these techniques will allow astronomers to successfully implement neural network models trained on simulation data to efficiently detect and study astrophysical objects in current and future large-scale astronomical surveys.
Domain adaptation techniques for improved cross-domain study of galaxy mergers
Ćiprijanović, A., Kafkes, D., Jenkins, S., Downey, K., Perdue, G. N., Madireddy, S., Johnston, T., Nord, B.
In astronomy, neural networks are often trained on simulated data with the prospect of being applied to real observations. Unfortunately, simply training a deep neural network on images from one domain does not guarantee satisfactory performance on new images from a different domain. The ability to share cross-domain knowledge is the main advantage of modern deep domain adaptation techniques. Here we demonstrate the use of two techniques -- Maximum Mean Discrepancy (MMD) and adversarial training with Domain Adversarial Neural Networks (DANN) -- for the classification of distant galaxy mergers from the Illustris-1 simulation, where the two domains presented differ only due to inclusion of observational noise. We show how the addition of either MMD or adversarial training greatly improves the performance of the classifier on the target domain when compared to conventional machine learning algorithms, thereby demonstrating great promise for their use in astronomy.