Diffusing Differentiable Representations

We introduce a novel, training-free method for sampling (diffreps) using pretrained diffusion models. Rather than merely mode-seeking, our method achieves sampling by pulling back the dynamics of the reverse-time process--from the image space to the diffrep parameter space--and updating the parameters according to this pulled-back process. We identify an implicit constraint on the samples induced by the diffrep and demonstrate that addressing this constraint significantly improves the consistency and detail of the generated objects.