The TAP free energy for high-dimensional linear regression

The analysis of high-dimensional probability distributio ns is a central challenge in modern Statistics and Machine Learning. This i s particularly true in the context of Bayesian Statistics, where scientists carry out inferen ce based on the posterior distribution. In modern applications, the posterior distribution is typi cally high-dimensional, and analytically intractable. V ariational Inference (VI) has emerge d as an attractive option to approximate these intractable distributions, facilitating fast, parallel computations in state-of-the-art applications [ 32, 10 ]. In this approach, the distribution of interest is approxi mated (in KL divergence) by distributions from a pre-specified, more tract able collection. The simplest version of VI is the Naive Mean-field approximation (NMF), where the distribution of interest is approximated by a product distribution.