Reviews: Automating Bayesian optimization with Bayesian optimization

Since the efficiency of Bayesian optimization mostly hinges on properly modelling the objective function, picking an appropriate model is essential. Usually, the model of choice is a Gaussian process with a simple Matern or SQE kernel, which is prone to model miss-specification. The proposed method extends the main loop in Bayesian optimization by an additional step to automatically select promising models based on the observed data. To solve this inner optimization problem, they use Bayesian optimization in model space to find a composition of kernels that account for the uncertainty of the objective function. Overall I do think that the method is sensible and addresses an important problem of Bayesian optimization, i.e model miss specification. Also, the paper is well written and clearly structured and I really enjoyed reading it.