Weshowthatthelearned model exhibits strong performances interms ofimage and text generation and anomaly detection.The one-pagecode can be found in supplementarymaterials.

Toeffectively tackle this issue and learn more expressiveprior models, wedevelop theadaptivemulti-stage density ratio estimation which breaks the estimation into multiple stages and learn different stages ofdensity ratiosequentially andadaptively. Thelatent priormodel canbe gradually learned using ratio estimated in previous stage so that the final latent spaceEBMpriorcanbenaturally formed byproduct ofratiosindifferentstages. The proposed method enables informativeand much sharper prior than existing baselines, and can be trained efficiently.

Vision transformer networks have shown superiority in many computer vision tasks.

We show that diffusion models actually are capable of generating high quality samples, sometimes better than the published results on other types of generative models (Section 4). In addition, we show that a certain parameterization of diffusion models reveals an equivalence with denoising score matching over multiple noise levels during training and with annealed Langevin dynamics during sampling (Section 3.2) [55, 61].