Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment

In this paper, we point out that suboptimal noise-data mapping leads to slow training of diffusion models. We emphasize that this random mixture of noise-data mapping complicates the optimization of the denoising function in diffusion models. Drawing inspiration from the immiscibility phenomenon in physics, we propose Immiscible Diffusion, a simple and effective method to improve the random mixture of noise-data mapping. In physics, miscibility can vary according to various intermolecular forces. Thus, immiscibility means that the mixing of molecular sources is distinguishable.