Stochastic Normalizing Flows

Normalizing flows (Rezende & Mohamed, 2015) are probabilistic models constructed as a sequence of successive transformations applied to some initial distribution. A key strength of normalizing flows is their expressive power as generative models, while enjoying an explicitly computable form of the likelihood function evaluated on the transformed space. This makes them especially well-equipped for variational inference (VI). Neural networks are often used as inspiration for finding effective transformations (Dinh et al., 2015; van den Berg et al., 2018). Continuous normalizing flows were later developed in Chen et al. (2018) as a means to perform maximum likelihood estimation and VI for large-scale probabilistic models derived from ordinary differential equations (ODEs).