Score-Based Generative Models Detect Manifolds

Score-based generative models (SGMs) need to approximate the scores abla \log p_t of the intermediate distributions as well as the final distribution p_T of the forward process. The theoretical underpinnings of the effects of these approximations are still lacking. We find precise conditions under which SGMs are able to produce samples from an underlying (low-dimensional) data manifold \mathcal{M} . This assures us that SGMs are able to generate the "right kind of samples". For example, taking \mathcal{M} to be the subset of images of faces, we provide conditions under which the SGM robustly produces an image of a face, even though the relative frequencies of these images might not accurately represent the true data generating distribution.