Supervised Learning
Elliptical Perturbations for Differential Privacy
We study elliptical distributions in locally convex vector spaces, and determine conditions when they can or cannot be used to satisfy differential privacy (DP). A requisite condition for a sanitized statistical summary to satisfy DP is that the corresponding privacy mechanism must induce equivalent probability measures for all possible input databases. We show that elliptical distributions with the same dispersion operator, C, are equivalent if the difference of their means lies in the Cameron-Martin space of C . In the case of releasing finite-dimensional summaries using elliptical perturbations, we show that the privacy parameter null can be computed in terms of a one-dimensional maximization problem. We apply this result to consider multivariate Laplace, t, Gaussian, and K -norm noise. Surprisingly, we show that the multivariate Laplace noise does not achieve null -DP in any dimension greater than one. Finally, we show that when the dimension of the space is infinite, no elliptical distribution can be used to give null -DP; only (null,ฮด)-DP is possible.
Deep Language Geometry: Constructing a Metric Space from LLM Weights
Shamrai, Maksym, Hamolia, Vladyslav
We introduce a novel framework that utilizes the internal weight activations of modern Large Language Models (LLMs) to construct a metric space of languages. Unlike traditional approaches based on hand-crafted linguistic features, our method automatically derives high-dimensional vector representations by computing weight importance scores via an adapted pruning algorithm. Our approach captures intrinsic language characteristics that reflect linguistic phenomena. We validate our approach across diverse datasets and multilingual LLMs, covering 106 languages. The results align well with established linguistic families while also revealing unexpected inter-language connections that may indicate historical contact or language evolution. The source code, computed language latent vectors, and visualization tool are made publicly available at https://github.com/mshamrai/deep-language-geometry.
Conic Formulations of Transport Metrics for Unbalanced Measure Networks and Hypernetworks
Oliver, Mary Chriselda Antony, Hartman, Emmanuel, Needham, Tom
The Gromov-Wasserstein (GW) variant of optimal transport, designed to compare probability densities defined over distinct metric spaces, has emerged as an important tool for the analysis of data with complex structure, such as ensembles of point clouds or networks. To overcome certain limitations, such as the restriction to comparisons of measures of equal mass and sensitivity to outliers, several unbalanced or partial transport relaxations of the GW distance have been introduced in the recent literature. This paper is concerned with the Conic Gromov-Wasserstein (CGW) distance introduced by S ejourn e, Vialard, and Peyr e [35]. We provide a novel formulation in terms of semi-couplings, and extend the framework beyond the metric measure space setting, to compare more general network and hypernetwork structures. With this new formulation, we establish several fundamental properties of the CGW metric, including its scaling behavior under dilation, variational convergence in the limit of volume growth constraints, and comparison bounds with established optimal transport metrics. We further derive quantitative bounds that characterize the robustness of the CGW metric to perturbations in the underlying measures. The hypernetwork formulation of CGW admits a simple and provably convergent block coordinate ascent algorithm for its estimation, and we demonstrate the computational tractability and scalability of our approach through experiments on synthetic and real-world high-dimensional and structured datasets.