Gradient estimators for normalising flows

Expressed in a form of an algorithm applied to study a simple classical statistical mechanics problem by Metropolis et al. [1] it is ubiquitous as a tool of dealing with complicated probability distributions (see for example [2]). In many cases one resorts to the construction of an associated Markov chain of consecutive proposals which provides a mathematically grounded way of generating samples from a given distribution even when the proper normalization of the latter is not known. The only limiting factor of the approach is the statistical uncertainty which directly depends on the number of statistically independent configurations. Hence, the effectiveness of any such simulation algorithm can be linked to its autocorrelation time which quantifies how many configurations are produced before a new, statistically independent configuration appears. For systems close to phase transitions the increasing autocorrelation times, a phenomenon called critical slowing down, is usually the main factor which limits the statistical precision of outputs. The recent interest in machine learning techniques has offered possible ways of dealing with this problem. Ref. [3] and later Ref. [4] proposed autoregressive neural networks as a mechanism of generating independent configurations which can be used as proposals in the construction of the Markov chain. The new algorithm was hence called Neural Markov Chain Monte Carlo (NMCMC). Once the neural network is sufficiently well trained one indeed finds that autocorrelation times are significantly reduced as was demonstrated in the context of the two-dimensional Ising model in Ref. [5].