Spread Divergences

For distributions p and q with different support, the divergence D(p q) generally will not exist. We define a spread divergence D(p q) on modified p and q and describe sufficient conditions for the existence of such a divergence. We give examples of using a spread divergence to train implicit generative models, including linearmodels (Principal Components Analysis and Independent Components Analysis) and nonlinear models (Deep Generative Networks).