Variational Inference for Latent Variable Models in High Dimensions

In modern applications, these models typically involve a large number of parameters and latent variables, resulting in complex and high-dimensional posteriors that are computationally intractable. For such scenarios, traditional Markov chain Monte Carlo (MCMC) approaches often suffer from lengthy burn-in periods and generally lack scalability [11]. Recently, variational inference (VI) [31, 10, 52, 11] has emerged as a popular and scalable alternative method for approximating intractable posterior distributions in large-scale applications (where the number of observations and dimensionality are both large) and is typically orders of magnitude faster than MCMC methods. Among the various forms of VI, arguably the most widely used and important is mean-field variational inference (MFVI) [52, 11], which approximates the intractable posterior by a product distribution. This approach has been widely adopted in statistics and machine learning, thanks to efficient algorithmic implementations based on coordinate ascent variational inference (CAVI) [10, 11, 19, 7, 5, 36, 14, 34].