Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?

We now have rigorous convergence guarantees that, for certain well-behaved posteriors, BBVI achieves a convergence rate of (1), corresponding We prove that black-box variational inference to a computational complexity of (1)(Domke et al., (BBVI) with control variates, particularly 2023a; Kim et al., 2023b). A remaining theoretical question the sticking-the-landing(STL) estimator, is whether BBVI can achieve better rates, in particular converges at a geometric (traditionally called geometric convergence rates, which is traditionally "linear") rate under perfect variational family called "linear" convergence in the optimization literature specification. In particular, we prove a (see the textbook by Nesterov 2004, 1.2.3), correspondingtoacomplexityof(log(1)).