Connective Viewpoints of Signal-to-Noise Diffusion Models

Diffusion models (DM) have become a fundamental part of generative models, which excel in various domains, including creating images, generating audio, and interpolating complex data. The foundational framework for these models was introduced by Sohl-Dickstein et al. (2015), and Ho et al. (2020) further refined it with Denoising Diffusion Probabilistic Models (DDPMs). DDPMs add noise to data iteratively and learn to reverse this process, allowing them to model data distributions effectively. Signal-to-Noise (S2N) diffusion models Kingma and Gao (2024); Kingma et al. (2021) constitute an extensive class of diffusion models encompassing various other models such as variance-preserving (VP) and variance-exploding (VE) DM Song et al. (2020b), iDDPM Nichol and Dhariwal (2021), DDPM Ho et al. (2020), EDM Karras et al. (2022), and continuous variation models Kingma and Gao (2024); Kingma et al. (2021). Numerous efforts have been made to study Signal-to-Noise diffusion models from various perspectives. Notably, Kingma et al. (2021) began with a discrete S2N diffusion model, developed its variational-based backward inference, and finally examined the asymptotic behavior as the number of time steps approaches infinity, resulting in a continuous variational DM.