Neural Information Processing Systems http://nips.cc/

Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate.

Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate.

Neural Information Processing Systems http://nips.cc/

We would like to thank the reviewers for their kind and thoughtful comments. Any attempt to mitigate particle degeneracy (e.g. Replicating Fig 1BD, we find similar, albeit less extreme results, with TMC always being faster than SMC. In particular, we have included Eq. 36 in the main text, and also included the corresponding choice of This should help to clarify that Eq. 11 applies to any directed graphical model (we have also included references In the example in Figure 1, we consider a model that does not have a chain-structure (see Appendix Figure 1A). IW AE performs arbitrarily badly due to the high-dimensionality of the state-space.

Importance sampling, which involves sampling from a probability density function (PDF) proportional to the product of an importance weight function and a base PDF, is a powerful technique with applications in variance reduction, biased or customized sampling, data augmentation, and beyond. Inspired by the growing availability of score-based generative models (SGMs), we propose an entirely training-free Importance sampling framework that relies solely on an SGM for the base PDF. Our key innovation is realizing the importance sampling process as a backward diffusion process, expressed in terms of the score function of the base PDF and the specified importance weight function--both readily available--eliminating the need for any additional training. We conduct a thorough analysis demonstrating the method's scalability and effectiveness across diverse datasets and tasks, including importance sampling for industrial and natural images with neural importance weight functions. The training-free aspect of our method is particularly compelling in real-world scenarios where a single base distribution underlies multiple biased sampling tasks, each requiring a different importance weight function. To the best of our knowledge our approach is the first importance sampling framework to achieve this.