Optimization
Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization
We consider the penalized distributionally robust optimization (DRO) problem with a closed, convex uncertainty set, a setting that encompasses learning using f -DRO and spectral/ L-risk minimization.
Nonparametric Evaluation of Noisy ICA Solutions
Independent Component Analysis (ICA) was introduced in the 1980's as a model
Why Transformers Need Adam: A Hessian Perspective
CNNs, MLPs, and quadratic problems, and find that SGD can perform on par with Adam on problems without block heterogeneity, but performs worse than Adam when the heterogeneity exists.
On Causal Discovery in the Presence of Deterministic Relations
In this paper, we find, supported by both theoretical analysis and empirical evidence, that score-based methods with exact search can naturally address the issues of deterministic relations under rather mild assumptions. Nonetheless, exact score-based methods can be computationally expensive.