Goto

Collaborating Authors

 ncsn






Review for NeurIPS paper: Improved Techniques for Training Score-Based Generative Models

Neural Information Processing Systems

Weaknesses: The experimental section does not quite have the experiments I was hoping for. I was hoping to understand *which* techniques were important to scale the model to higher-resolution images. I know that using all techniques are needed for the model to learn LSUN images, but my suspicion is that only a subset of them (perhaps only EMA) is needed for regular NCSN to learn high-resolution images. Better understanding of which techniques are important would help explain what is needed for scaling the model. Moreover, for the ablation experiments provided, it does not seem that all 5 techniques are need for all datasets. Fig 5 seems to show that the simplified network is not needed on CelebA 64x64 as FID is better using the original network.


Review for NeurIPS paper: Denoising Diffusion Probabilistic Models

Neural Information Processing Systems

However, the empirical performance of the proposed approach shows huge advantage over NCSN. Can the author elaborate what makes this difference? To my knowledge, the difference are The number of noise-levels (denoted as L): For the diffusion model, L 1000. The scheduling sequence of variance (denoted as beta_t, which is the \sigma 2 in NCSN): For the diffusion model, beta_1 1e-4, beta_T 0.02, and linear schedule is employed. For NCSN, they consider the geometric sequence, and beta_T is much larger for NCSNv2.


Review for NeurIPS paper: Denoising Diffusion Probabilistic Models

Neural Information Processing Systems

The paper gives insights on DSM (Denoising Score Matching) and MCMC method and links it to Probabilistic Diffusion models. This is novel and reviewer agrees that the paper has a good contribution. Concerns: • Algorithmically it is the same algorithm of NCSN with 1) different hyper-parameters motivated from diffusion models ( like scaling of inputs between stages) 2) different architectural choices • The FID is very low, maybe some memorization? Please include in the final version of the paper all the details in answers in rebuttal to R2 on the main comparison with NCSN, architecture choices etc, training time, sampling time, the need for cross-validation etc and how long the full training and cross validation takes. While probabilistic diffusion models are elegant their compute time is intensive please discuss this in the paper, and how you think this can be addressed.


Source Separation with Deep Generative Priors

Jayaram, Vivek, Thickstun, John

arXiv.org Machine Learning

Despite substantial progress in signal source separation, results for richly structured data continue to contain perceptible artifacts. In contrast, recent deep generative models can produce authentic samples in a variety of domains that are indistinguishable from samples of the data distribution. This paper introduces a Bayesian approach to source separation that uses generative models as priors over the components of a mixture of sources, and Langevin dynamics to sample from the posterior distribution of sources given a mixture. This decouples the source separation problem from generative modeling, enabling us to directly use cutting-edge generative models as priors. The method achieves state-of-the-art performance for MNIST digit separation. We introduce new methodology for evaluating separation quality on richer datasets, providing quantitative evaluation of separation results on CIFAR-10. We also provide qualitative results on LSUN.