Ordinal Bayesian Optimisation

In BO, nonparametric Gaussian processes (GPs) provide flexible and fast-to-evaluate surrogates of the objective functions. Sequential design decisions, so-called acquisitions, judiciously balance exploration and exploitation in search for global optima, leveraging the uncertainty estimates provided by the GP posterior distributions (see Mockus et al. (1978); Jones et al. (1998) for early works or Shahriari et al. (2015) for a recent review). One of the weaknesses of vanilla BO lies in the underlying assumption that the objective function is a realisation of a GP: when this assumption is strongly violated, the GP model is weakly predictive and BO becomes inefficient. Two classical examples where BO fails are ill-conditioned problems, when the objective function has strong variations on the domain boundaries but is very flat in its central region (or conversely), and non-Lipschitz objectives, for instance with local discontinuities. High conditioning is typical in "exploratory" optimisation problems, when the parameter space is initially chosen very large. Discontinuities are frequent in computational fluid dynamics problems for instance, where a small change in the parameters results in a change of physics (e.g.