Ordinal Bayesian Optimisation

arXiv.org Machine Learning

In BO, nonparametric Gaussian processes (GPs) provide flexible and fast-to-evaluate surrogates of the objective functions. Sequential design decisions, so-called acquisitions, judiciously balance exploration and exploitation in search for global optima, leveraging the uncertainty estimates provided by the GP posterior distributions (see Mockus et al. (1978); Jones et al. (1998) for early works or Shahriari et al. (2015) for a recent review). One of the weaknesses of vanilla BO lies in the underlying assumption that the objective function is a realisation of a GP: when this assumption is strongly violated, the GP model is weakly predictive and BO becomes inefficient. Two classical examples where BO fails are ill-conditioned problems, when the objective function has strong variations on the domain boundaries but is very flat in its central region (or conversely), and non-Lipschitz objectives, for instance with local discontinuities. High conditioning is typical in "exploratory" optimisation problems, when the parameter space is initially chosen very large. Discontinuities are frequent in computational fluid dynamics problems for instance, where a small change in the parameters results in a change of physics (e.g.

Integrating Bayesian Networks into Knowledge-Intensive CBR

AAAI Conferences

The role of the general domain knowledge is to explain why two cases are similar based on semantic and pragmatic criteria, how a past solution may be adapted to a new problem case, and/or what to retain from a case just solved (Porter et.

Quantifying uncertainties on excursion sets under a Gaussian random field prior

arXiv.org Machine Learning

We focus on the problem of estimating and quantifying uncertainties on the excursion set of a function under a limited evaluation budget. We adopt a Bayesian approach where the objective function is assumed to be a realization of a Gaussian random field. In this setting, the posterior distribution on the objective function gives rise to a posterior distribution on excursion sets. Several approaches exist to summarize the distribution of such sets based on random closed set theory. While the recently proposed Vorob'ev approach exploits analytical formulae, further notions of variability require Monte Carlo estimators relying on Gaussian random field conditional simulations. In the present work we propose a method to choose Monte Carlo simulation points and obtain quasi-realizations of the conditional field at fine designs through affine predictors. The points are chosen optimally in the sense that they minimize the posterior expected distance in measure between the excursion set and its reconstruction. The proposed method reduces the computational costs due to Monte Carlo simulations and enables the computation of quasi-realizations on fine designs in large dimensions. We apply this reconstruction approach to obtain realizations of an excursion set on a fine grid which allow us to give a new measure of uncertainty based on the distance transform of the excursion set. Finally we present a safety engineering test case where the simulation method is employed to compute a Monte Carlo estimate of a contour line.

Batched High-dimensional Bayesian Optimization via Structural Kernel Learning

arXiv.org Machine Learning

Optimization of high-dimensional black-box functions is an extremely challenging problem. While Bayesian optimization has emerged as a popular approach for optimizing black-box functions, its applicability has been limited to low-dimensional problems due to its computational and statistical challenges arising from high-dimensional settings. In this paper, we propose to tackle these challenges by (1) assuming a latent additive structure in the function and inferring it properly for more efficient and effective BO, and (2) performing multiple evaluations in parallel to reduce the number of iterations required by the method. Our novel approach learns the latent structure with Gibbs sampling and constructs batched queries using determinantal point processes. Experimental validations on both synthetic and real-world functions demonstrate that the proposed method outperforms the existing state-of-the-art approaches.