Goto

Collaborating Authors

 semidefinite program


Smoothed analysis of the low-rank approach for smooth semidefinite programs

Thomas Pumir, Samy Jelassi, Nicolas Boumal

Neural Information Processing Systems

Inprior work, ithas been shown that, when the constraints on the factorized variable regularly define a smooth manifold, providedk is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading tothequestion: aresuch points also approximately optimal?






Optimal Transportation and Alignment Between Gaussian Measures

Dandapanthula, Sanjit, Podkopaev, Aleksandr, Kasiviswanathan, Shiva Prasad, Ramdas, Aaditya, Goldfeld, Ziv

arXiv.org Artificial Intelligence

Optimal transport (OT) and Gromov-Wasserstein (GW) alignment provide interpretable geometric frameworks for comparing, transforming, and aggregating heterogeneous datasets -- tasks ubiquitous in data science and machine learning. Because these frameworks are computationally expensive, large-scale applications often rely on closed-form solutions for Gaussian distributions under quadratic cost. This work provides a comprehensive treatment of Gaussian, quadratic cost OT and inner product GW (IGW) alignment, closing several gaps in the literature to broaden applicability. First, we treat the open problem of IGW alignment between uncentered Gaussians on separable Hilbert spaces by giving a closed-form expression up to a quadratic optimization over unitary operators, for which we derive tight analytic upper and lower bounds. If at least one Gaussian measure is centered, the solution reduces to a fully closed-form expression, which we further extend to an analytic solution for the IGW barycenter between centered Gaussians. We also present a reduction of Gaussian multimarginal OT with pairwise quadratic costs to a tractable optimization problem and provide an efficient algorithm to solve it using a rank-deficiency constraint. To demonstrate utility, we apply our results to knowledge distillation and heterogeneous clustering on synthetic and real-world datasets.



Exploiting Tradeoffs for Exact Recovery in Heterogeneous Stochastic Block Models

Amin Jalali, Qiyang Han, Ioana Dumitriu, Maryam Fazel

Neural Information Processing Systems

The Stochastic Block Model (SBM) is a widely used random grap h model for networks with communities. Despite the recent burst of inte rest in community detection under the SBM from statistical and computational points of view, there are still gaps in understanding the fundamental limits of re covery. In this paper, we consider the SBM in its full generality, where there is no r estriction on the number and sizes of communities or how they grow with the numb er of nodes, as well as on the connectivity probabilities inside or across c ommunities. For such stochastic block models, we provide guarantees for exact re covery via a semidef-inite program as well as upper and lower bounds on SBM paramet ers for exact recoverability. Our results exploit the tradeoffs among th e various parameters of heterogenous SBM and provide recovery guarantees for man y new interesting SBM configurations.


Smoothed analysis of the low-rank approach for smooth semidefinite programs

Neural Information Processing Systems

We consider semidefinite programs (SDPs) of size $n$ with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix $Y$ of size $n\times k$ such that $X=YY^*$ is the SDP variable. The advantages of such formulation are twofold: the dimension of the optimization variable is reduced, and positive semidefiniteness is naturally enforced. However, optimization in $Y$ is non-convex. In prior work, it has been shown that, when the constraints on the factorized variable regularly define a smooth manifold, provided $k$ is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading to the question: are such points also approximately optimal? To this end, and under similar assumptions, we use smoothed analysis to show that approximate SOSPs for a randomly perturbed objective function are approximate global optima, with $k$ scaling like the square root of the number of constraints (up to log factors).