Lookahead Bayesian Optimization via Rollout: Guarantees and Sequential Rolling Horizons

We consider the optimization problem: x arg max x X f (x), (1) where x is a d-dimensional vector and X is a compact (closed and bounded) set in R d . Given limited budget B, BO aims to search for the optimal x by itera-tively updating a surrogate model of f (x), where this surrogate is used to find the next design to evaluate. Typically, in BO, the surrogate model is a Gaussian process ( GP), due to its Bayesian interpretation and uncertainty quantification capability (see Rasmussen (2003) for more information). More specifically, given the current data D k, BO aims to determine the next informative sampling point x k 1 by solving the auxiliary problem: x k 1: x arg max x X Q k(x; D k). (2) where Q k is a acquisition/utility function that only involves evaluating the surrogate and not the expensive objective function f . Typically, evaluation of acquisition function is relatively cheap. The rationale is to seek design points that produce maximum increment of the objective function. After Eq. (2) is solved, the iterative algorithm proceeds by augmenting the current training data D k with a new observation to obtain D k 1 D k { (x k 1,y k 1) }. Popular choices of acquisition functions are entropy search (ES) (Hennig and Schuler, 2012), predictive entropy search (PES) (Hern andez-Lobato et al., 2014) and expectation improvement (EI) (Lam et al., 2016). All aforementioned functions exploit myopic strategies and ignore the future information.