FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation

An SGM involves a Score-based generative models (SGMs) learn a stochastic forward and backward process. In the forward family of noise-conditional score functions corresponding process, also known as the diffusion process, noise with to the data density perturbed with gradually increasing variances is added to each data point increasingly large amounts of noise. These until the original structure is lost, transforming data into perturbed data densities are linked together by pure noise. The backward process attempts to reverse the the Fokker-Planck equation (FPE), a partial differential diffusion process by using a neural network (called a noiseconditional equation (PDE) governing the spatialtemporal score model) that is trained to gradually denoise evolution of a density undergoing a diffusion the data, effectively transforming pure noise into clean data process. In this work, we derive a corresponding samples. The neural network is trained with a denoising equation called the score FPE that score matching objective (Hyvärinen & Dayan, 2005; Vincent, characterizes the noise-conditional scores of the 2011) to estimate the score (i.e., the gradient of the perturbed data densities (i.e., their gradients). Surprisingly, log-likelihood function) of the data density perturbed with despite the impressive empirical performance, various amounts of noise (as in forward process).