2 Preliminaries Wefirstintroduce theproblem setting andthelocal Bayesianoptimization framework. Weaimto numericallysolveoptimizationproblemsoftheform: givenx0 D,findx =argmin

We demonstrate that, surprisingly, the expected value ofthegradient isnotalwaysthedirection maximizing theprobability ofdescent, and in fact, these directions may be nearly orthogonal. This observation then inspires an elegant optimization scheme seeking to maximize the probability of descent while moving in the direction of most-probable descent.