adversarial energy-based model
Improving Adversarial Energy-Based Model via Diffusion Process
Geng, Cong, Han, Tian, Jiang, Peng-Tao, Zhang, Hao, Chen, Jinwei, Hauberg, Søren, Li, Bo
MCMC-based EBMs (Du & Mordatch, 2019; Nijkamp et al., 2019) evaluate Generative models have shown strong generation the gradient of the objective through Markov chain ability while efficient likelihood estimation is less Monte Carlo (MCMC) sampling on the defined energy function, explored. Energy-based models (EBMs) define which can be computationally expensive for both training a flexible energy function to parameterize unnormalized and sampling. Adversarial EBMs (Grathwohl et al., densities efficiently but are notorious for 2021; Geng et al., 2021) introduce a generator to form a being difficult to train. Adversarial EBMs introduce minimax game between alternative optimization of this generator a generator to form a minimax training game and energy function, allowing for MCMC-free EBM to avoid expensive MCMC sampling used in traditional training and fast sampling. EBMs, but a noticeable gap between adversarial EBMs and other strong generative models Although adversarial EBMs have great potential in distribution still exists. Inspired by diffusion-based models, modeling, they still have some limitations that can we embedded EBMs into each denoising step to be mainly attributed to three reasons. First, as is pointed split a long-generated process into several smaller out in Mescheder et al. (2018) and Geng et al. (2021), minimax steps. Besides, we employ a symmetric Jeffrey divergence training can be unstable if two alternative optimization and introduce a variational posterior distribution steps are not well balanced. This instability poses a significant for the generator's training to address the challenge in fitting the marginal energy distribution main challenges that exist in adversarial EBMs.