Connections between Support Vector Machines, Wasserstein distance and gradient-penalty GANs

arXiv.org Machine Learning

We generalize the concept of maximum-margin classifiers (MMCs) to arbitrary norms and non-linear functions. Support Vector Machines (SVMs) are a special case of MMC. We find that MMCs can be formulated as Integral Probability Metrics (IPMs) or classifiers with some form of gradient norm penalty. This implies a direct link to a class of Generative adversarial networks (GANs) which penalize a gradient norm. We show that the Discriminator in Wasserstein, Standard, Least-Squares, and Hinge GAN with Gradient Penalty is an MMC. We explain why maximizing a margin may be helpful in GANs. We hypothesize and confirm experimentally that $L^\infty$-norm penalties with Hinge loss produce better GANs than $L^2$-norm penalties (based on common evaluation metrics). We derive the margins of Relativistic paired (Rp) and average (Ra) GANs.


Connections between SVMs, Wasserstein distance and GANs

#artificialintelligence

Check out my new paper entitled "Support Vector Machines, Wasserstein's distance and gradient-penalty GANs are connected"! In this paper, we explain how one can derive SVMs and gradient penalized GANs (or those with Lipschitz-1 discriminator) from the same framework! We also show new gradient penalties that lead to better GANs. This paper may completely change your perspective on the Wasserstein's distance, Wasserstein GAN (WGAN), Hinge GAN (HingeGAN), and the use of gradient penalties in GANs. At least, it did for me!


Chiefs' Peters Fined $24,309 for Penalty Flag Toss

U.S. News

During the Jets' go-ahead drive in the fourth quarter, the Chiefs were called for three penalties to prolong the possession. The last of the penalties frustrated Peters, who picked up one of the flags and tossed it into the seats at MetLife Stadium. Peters was penalized for unsportsmanlike conduct, and he headed straight to the locker room thinking he had been ejected.


Ducks are getting pretty acquainted with the penalty box

Los Angeles Times

The Ducks can't stay out of the penalty box. They incurred 16 penalty minutes in Friday's 3-1 loss to the Avalanche in Colorado, with five minors in the first period. Four of those first-period penalties were slashing minors, keeping with the new league crackdown on stick infractions that "seem to be [intentional] or directed at the fingers and hands with greater force," as new NHL head of player safety George Parros described it. The fifth was a penalty for too many men on the ice, and the Ducks received another two slashing penalties later in the game. "Every time there's new rules, there's an adjustment part," said center Antoine Vermette, who has eight penalty minutes and registered 93 with the Arizona Coyotes two seasons ago.


Israel's new death penalty bill 'targets Palestinians'

Al Jazeera

The Israeli government's proposal to make it easier for judges to hand out the death penalty for "terrorist activity" has been condemned as "fascist" by Palestinian politicians and rights groups, who fear it will give Israel legal cover to target Palestinians.