Inversion-based Latent Bayesian Optimization