Statistical Learning
MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention
Wang, Tianyi, Fan, Jianan, Zhang, Dingxin, Liu, Dongnan, Xia, Yong, Huang, Heng, Cai, Weidong
Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease. Multi-modal self-supervised learning has demonstrated remarkable potential in learning pathological representations by integrating diverse data sources. Conventional multi-modal integration methods primarily emphasize modality alignment, while paying insufficient attention to retaining the modality-specific structures. However, unlike conventional scenarios where multi-modal inputs share highly overlapping features, histopathology and transcriptomics exhibit pronounced heterogeneity, offering orthogonal yet complementary insights. Histopathology provides morphological and spatial context, elucidating tissue architecture and cellular topology, whereas transcriptomics delineates molecular signatures through gene expression patterns. This inherent disparity introduces a major challenge in aligning them while maintaining modality-specific fidelity. To address these challenges, we present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention. MIRROR employs dedicated encoders to extract comprehensive features for each modality, which is further complemented by a modality alignment module to achieve seamless integration between phenotype patterns and molecular profiles. Furthermore, a modality retention module safeguards unique attributes from each modality, while a style clustering module mitigates redundancy and enhances disease-relevant information by modeling and aligning consistent pathological signatures within a clustering space. Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance, demonstrating its effectiveness in constructing comprehensive oncological feature representations and benefiting the cancer diagnosis.
The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction
Islam, Mohammad Tariqul, Fleischer, Jason W.
The current era is characterized by a deluge of high-dimensional data. Dimensionality reduction (DR) techniques have emerged as tools for exploratory analysis of such data by visualizing the underlying structure. The most popular methods, t-distributed stochastic neighbor embedding [1] and uniform manifold approximation and projection (UMAP) [2] are grounded in the attraction-repulsion dynamics that bring similar data points closer while pushing dissimilar ones apart. As unsupervised algorithms, these do not rely on labeled data; instead, they identify and preserve the intrinsic structure of high-dimensional data by leveraging local (attractive) and global (repulsive) relationships (forces). This makes these algorithms particularly well-suited for tasks such as clustering [3], exploratory data analysis [4], anomaly detection in semiconductor manufacturing [5], visual search [6], time series analysis [7], studying representation convergence [8], and outlier image detection [9], where visualizing hidden patterns in unlabeled data is critical and meaningful.
Incorporating Attributes and Multi-Scale Structures for Heterogeneous Graph Contrastive Learning
Jiang, Ruobing, Li, Yacong, Liu, Haobing, Yu, Yanwei
Heterogeneous graphs (HGs) are composed of multiple types of nodes and edges, making it more effective in capturing the complex relational structures inherent in the real world. However, in real-world scenarios, labeled data is often difficult to obtain, which limits the applicability of semi-supervised approaches. Self-supervised learning aims to enable models to automatically learn useful features from data, effectively addressing the challenge of limited labeling data. In this paper, we propose a novel contrastive learning framework for heterogeneous graphs (ASHGCL), which incorporates three distinct views, each focusing on node attributes, high-order and low-order structural information, respectively, to effectively capture attribute information, high-order structures, and low-order structures for node representation learning. Furthermore, we introduce an attribute-enhanced positive sample selection strategy that combines both structural information and attribute information, effectively addressing the issue of sampling bias. Extensive experiments on four real-world datasets show that ASHGCL outperforms state-of-the-art unsupervised baselines and even surpasses some supervised benchmarks.
On the clustering behavior of sliding windows
Alexeev, Boris, Luo, Wenyan, Mixon, Dustin G., Zhang, Yan X
Clustering is one of the most common tasks in data science, and given the ubiquity of timeseries data, one is naturally inclined to cluster it. In order to perform Euclidean clustering (such as k-means clustering) on timeseries data, one must first map the data into Euclidean space. This is traditionally accomplished with a sliding window.
Wasserstein-based Kernels for Clustering: Application to Power Distribution Graphs
Oneto, Alfredo, Gjorgiev, Blazhe, Sansavini, Giovanni
Many data clustering applications must handle objects that cannot be represented as vector data. In this context, the bag-of-vectors representation can be leveraged to describe complex objects through discrete distributions, and the Wasserstein distance can effectively measure the dissimilarity between them. Additionally, kernel methods can be used to embed data into feature spaces that are easier to analyze. Despite significant progress in data clustering, a method that simultaneously accounts for distributional and vectorial dissimilarity measures is still lacking. To tackle this gap, this work explores kernel methods and Wasserstein distance metrics to develop a computationally tractable clustering framework. The compositional properties of kernels allow the simultaneous handling of different metrics, enabling the integration of both vectors and discrete distributions for object representation. This approach is flexible enough to be applied in various domains, such as graph analysis and image processing. The framework consists of three main components. First, we efficiently approximate pairwise Wasserstein distances using multiple reference distributions. Second, we employ kernel functions based on Wasserstein distances and present ways of composing kernels to express different types of information. Finally, we use the kernels to cluster data and evaluate the quality of the results using scalable and distance-agnostic validity indices. A case study involving two datasets of 879 and 34,920 power distribution graphs demonstrates the framework's effectiveness and efficiency.
Model Predictive Path Integral Control of I2RIS Robot Using RBF Identifier and Extended Kalman Filter
Esfandiari, Mojtaba, Du, Pengyuan, Wei, Haochen, Gehlbach, Peter, Munawar, Adnan, Kazanzides, Peter, Iordachita, Iulian
Modeling and controlling cable-driven snake robots is a challenging problem due to nonlinear mechanical properties such as hysteresis, variable stiffness, and unknown friction between the actuation cables and the robot body. This challenge is more significant for snake robots in ophthalmic surgery applications, such as the Improved Integrated Robotic Intraocular Snake (I$^2$RIS), given its small size and lack of embedded sensory feedback. Data-driven models take advantage of global function approximations, reducing complicated analytical models' challenge and computational costs. However, their performance might deteriorate in case of new data unseen in the training phase. Therefore, adding an adaptation mechanism might improve these models' performance during snake robots' interactions with unknown environments. In this work, we applied a model predictive path integral (MPPI) controller on a data-driven model of the I$^2$RIS based on the Gaussian mixture model (GMM) and Gaussian mixture regression (GMR). To analyze the performance of the MPPI in unseen robot-tissue interaction situations, unknown external disturbances and environmental loads are simulated and added to the GMM-GMR model. These uncertainties of the robot model are then identified online using a radial basis function (RBF) whose weights are updated using an extended Kalman filter (EKF). Simulation results demonstrated the robustness of the optimal control solutions of the MPPI algorithm and its computational superiority over a conventional model predictive control (MPC) algorithm.
Operational Change Detection for Geographical Information: Overview and Challenges
Rapid evolution of territories due to climate change and human impact requires prompt and effective updates to geospatial databases maintained by the National Mapping Agency. This paper presents a comprehensive overview of change detection methods tailored for the operational updating of large-scale geographic databases. This review first outlines the fundamental definition of change, emphasizing its multifaceted nature, from temporal to semantic characterization. It categorizes automatic change detection methods into four main families: rule-based, statistical, machine learning, and simulation methods. The strengths, limitations, and applicability of every family are discussed in the context of various input data. Then, key applications for National Mapping Agencies are identified, particularly the optimization of geospatial database updating, change-based phenomena, and dynamics monitoring. Finally, the paper highlights the current challenges for leveraging change detection such as the variability of change definition, the missing of relevant large-scale datasets, the diversity of input data, the unstudied no-change detection, the human in the loop integration and the operational constraints. The discussion underscores the necessity for ongoing innovation in change detection techniques to address the future needs of geographic information systems for national mapping agencies.
A Relaxed Wasserstein Distance Formulation for Mixtures of Radially Contoured Distributions
Chen, Keyu, Wang, Zetian, Zhang, Yunxin
Recently, a Wasserstein-type distance for Gaussian mixture models has been proposed. However, that framework can only be generalized to identifiable mixtures of general elliptically contoured distributions whose components come from the same family and satisfy marginal consistency. In this paper, we propose a simple relaxed Wasserstein distance for identifiable mixtures of radially contoured distributions whose components can come from different families. We show some properties of this distance and that its definition does not require marginal consistency. We apply this distance in color transfer tasks and compare its performance with the Wasserstein-type distance for Gaussian mixture models in an experiment. The error of our method is more stable and the color distribution of our output image is more desirable.
Reliable uncertainty quantification for 2D/3D anatomical landmark localization using multi-output conformal prediction
Jonkers, Jef, Coopman, Frank, Duchateau, Luc, Van Wallendael, Glenn, Van Hoecke, Sofie
Automatic anatomical landmark localization in medical imaging requires not just accurate predictions but reliable uncertainty quantification for effective clinical decision support. Current uncertainty quantification approaches often fall short, particularly when combined with normality assumptions, systematically underestimating total predictive uncertainty. This paper introduces conformal prediction as a framework for reliable uncertainty quantification in anatomical landmark localization, addressing a critical gap in automatic landmark localization. We present two novel approaches guaranteeing finite-sample validity for multi-output prediction: Multi-output Regression-as-Classification Conformal Prediction (M-R2CCP) and its variant Multi-output Regression to Classification Conformal Prediction set to Region (M-R2C2R). Unlike conventional methods that produce axis-aligned hyperrectangular or ellipsoidal regions, our approaches generate flexible, non-convex prediction regions that better capture the underlying uncertainty structure of landmark predictions. Through extensive empirical evaluation across multiple 2D and 3D datasets, we demonstrate that our methods consistently outperform existing multi-output conformal prediction approaches in both validity and efficiency. This work represents a significant advancement in reliable uncertainty estimation for anatomical landmark localization, providing clinicians with trustworthy confidence measures for their diagnoses. While developed for medical imaging, these methods show promise for broader applications in multi-output regression problems.
Empirical risk minimization algorithm for multiclass classification of S.D.E. paths
Denis, Christophe, Mintsa, Eddy Ella
Functional data analysis (Ramsay & Silverman, 2005) is an active field of research, as technological progress has facilitated the large-scale collection of such data in a wide range of fields, including biology (Crow, 2017)), physics (Romanczuk et al., 2012)), and finance (Lamberton & Lapeyre, 2011)). In particular, the classification of functional data is at the core of recent research efforts (Ismail Fawaz et al., 2019; Xiao et al., 2022). A specific case arises when the data are assumed to be generated by diffusion processes. The literature on statistical methods for stochastic differential equations has generated significant attention.