GAN and VAE from an Optimal Transport Point of View

This short article revisits some of the ideas introduced in [1] and [4] in a simple setup. "pushes forward" each elementary mass of a measure ζ in P(Z) by applying the map g to obtain then a mass in X, to build on aggregate a Because (1) is a linear program, it has a dual formulation, known as the Kantorovich problem [13, Thm. A key remark in Kantorovich's formulation is to notice that the cost of any pair (h, h) can always be improved by replacing h in (2) by the c-transform h As a side-note, and as previously commented in the literature, there is at this point no empirical evidence that supports the idea that using discriminative deep networks that way can result in accurate approximations of Wasserstein distances. These alternative formulations provide instead a very useful proxy for a quantity directly related to the Wasserstein distance. This is advantageous because now π is defined over Z X, which is lowerdimensional than X X, and also because, as in Equation (2), θ does not appear in the constraints either.