Optimal Transport Relaxations with Application to Wasserstein GANs

Optimal transport costs, which include the Wasserstein Distance and the Earth-Mover-Distance as special cases, have become useful tools in machine learning and statistics [16, 3, 1, 18, 8, 6]. The optimal transport cost between two distributions is computed (in its primal form) as a minimization problem, in which the cost of transporting one distribution to another is minimized over all possible joint distributions, leading to linear program (see for example,[26]). Optimal transport provides great flexibility when comparing (probability) measures and histograms. The transportation cost function (which we refer to as the cost function) can be used to capture key geometric characteristics [16]. It can be also used to compare discrete vs continuous distributions directly, without introducing smoothing, in contrast to alternatives such as the Kullback-Leibler divergence (see [3, 13] for more details). Also, by judiciously choosing the cost function, a Wasserstein distance can generate either the topology corresponding to weak convergence or the total variation distance. In data-driven applications, one needs to estimate the optimal transport cost by means of sampled data.