Supplemental Material for Wavelet Flow: Fast Training of High Resolution Normalizing Flows Jason J. Yu1,3 and Marcus A. Brubaker

In this section, we describe the annealed sampling process. First, we describe how MCMC can be used to draw samples from any distribution constructed as a normalizing flow. Next, we describe how the Wavelet Flow structure in particular is used to enable faster sampling. B.1 MCMC on an Annealed Flow The target distribution for MCMC is the annealed normalizing flow and can be written as: π Since we know the form of the density is closely related to a known normalizing flow, we can use the inverse of this flow, g, to reparameterize the density such that it becomes exactly Gaussian (and hence easier to sample) when γ = 1. For γ 1 the geometry should still be close to Gaussian and hence easier to sample from, particularly with values of γ close to 1. Reparameterizing the annealed distribution in terms of z gives: π In practice, we found that sampling in terms of z using the NUTS algorithm [3] is more efficient and can be done with a larger step size and fewer divergences, compared to sampling in terms of x.