What are Normalizing Flows?

Normalizing flows is a technique used in machine learning to build complex distributions from simple distributions. They have been applied in the context of generative modelling. They have become popular recently, and have received quite a lot of attention -- for example Glow, by OpenAI -- because of their immense power to model probability distributions. Suppose we have a continuous random variable z with some simple distribution like isotropic Gaussian distribution allows for easy sampling and density evaluation. The key idea is to transform this simple distribution with some function f into a more complicated one, we formulate f as a composition of sequence of invertible transformations so that overall transformation is also invertible.