Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models in computer graphics, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks which control how the model makes random choices based on the output it has generated thus far. We call such models neurally-guided procedural models. As a pre-computation, we train these models to maximize the likelihood of example outputs generated via SMC. They are then used as efficient SMC importance samplers, generating high-quality results with very few samples. We evaluate our method on L-system-like models with image-based constraints. Given a desired quality threshold, neurally-guided models can generate satisfactory results up to 10x faster than unguided models.

I think this paper is clearly written and makes some reasonable contributions, but could do with an editing pass to frame the approach a bit better and relate it to recent contributions that similarly seek to amortize inference in sequential models by training neural net proposals. While the authors are the first (to my knowledge) to train NN proposals for SMC in a procedural graphics / probabilistic programming setting, their approach is of course very closely related the one developed by Gu and colleagues [1], which the authors cite and the one proposed by Paige and Wood [2], which they do not. I am assuming this work was done more or less concurrently and independently, and I in principle don't see a problem for this paper from the point of view of novelty. That said, the paper in its current revision still reads a bit like a graphics paper that pitches using NN proposals for SMC as its core idea. This is unfortunate, in that it would have been nice to see the authors relate their work to that done by others.

Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models in computer graphics, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks which control how the model makes random choices based on the output it has generated thus far. We call such models neurally-guided procedural models. As a pre-computation, we train these models to maximize the likelihood of example outputs generated via SMC.