Ambiguity set and learning via Bregman and Wasserstein

Comparing probability distributions has been a recurring theme in many research areas of machine learning. In distribution learning, for example, one is interested in approximating the true distribution by an element in a predetermined class of probability distributions, and this element is chosen based on the observed data. Such choices rely on the divergence used in comparing distributions. While there is an abundance in statistical divergences, there is no consensus about the "ideal" way to measure the difference between distributions. In the theory of robust optimization, optimization problems are formulated under appropriate uncertainty sets for the model parameters and/or for the solutions against a certain measure of robustness. For instance, tractable uncertainty sets can be formulated in terms of chance constraints and expectation constraints under a given distribution P Jiang and Guan [2012]. However, when the distribution P itself is unknown, which is the usual scenario in most data-driven research, the concept of ambiguity set is introduced Bayraksan and Love [2015]. Thus, instead of optimizing under one particular distribution and under a deterministic set, distributionally robust stochastic optimization, aka DRSO, formulates optimization problems with a set of possible distributions, under the concept of ambiguity set.