Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithms

The task of drawing samples efficiently from high-dimensional complex probability distributions enables us to perform inference using complex statistical models from large amounts of data, where uncertainty quantification is of paramount importance to understand the intrinsic risk associated with every decision made with models learned from data. The ability to quantify uncertainty when comparing a theoretical or computational model to observations is critical to conducting a sound scientific investigation, particularly in machine-learned models and in the physical sciences like physics [92]. More specifically, Bayesian inference [96, 184] is a prominent method for linking models and observations and estimating uncertainties, in which sampling techniques are widely adopted, which also finds applications to various areas such as imaging processing and inverse problems (see e.g., [87]), and Bayesian neural networks and deep learning [134], etc. While Markov chain Monte Carlo (MCMC) methods [164] have been the major workhorse of such sampling tasks, most traditional MCMC algorithms were regarded as unscalable to high dimensions. In particular, in modern large-scale applications such as Bayesian deep learning in the overparameterized regime in which we want to make posterior inference on the neural network weights, traditional MCMC algorithms become computationally prohibitive in such high dimensions and alternative approaches such as variational inference (VI; see e.g., [21]) have been widely adopted.