Optimal Transport, CycleGAN, and Penalized LS for Unsupervised Learning in Inverse Problems

O PTIMAL T RANSPORT, C YCLEGAN, AND P ENALIZED LS FOR U NSUPERVISEDL EARNING IN I NVERSE P ROB-LEMS Byeongsu Sim 1 Gyutaek Oh 2 Sungjun Lim 2 Jong Chul Y e 1,2 1 Department of Mathematical Sciences, KAIST, Daejeon, Republic of Korea 2 Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea A BSTRACT The penalized least squares (PLS) is a classic approach to inverse problems, where a regularization term is added to stabilize the solution. Optimal transport (OT) is another mathematical framework for computer vision tasks by providing means to transport one measure to another at minimal cost. Cycle-consistent generative adversarial network (cycleGAN) is a recent extension of GAN to learn target distributions with less mode collapsing behavior. Although similar in that no supervised training is required, the algorithms look different, so the mathematical relationship between these approaches is not clear. In this article, we provide an important advance to unveil the missing link. Specifically, we reveal that a cycle-GAN architecture can be derived as a dual formulation of the optimal transport problem, if the PLS with a deep learning penalty is used as a transport cost between the two probability measures from measurements and unknown images. This suggests that cycleGAN can be considered as stochastic generalization of classical PLS approaches. Our derivation is so general that various types of cy-cleGAN architecture can be easily derived by merely changing the transport cost.