SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality

Optimization problems are ubiquitous in various fields, ranging from computer science and engineering to finance and healthcare. Whether the focus is on minimizing costs or improving efficiency, these challenges frequently involve finding the best outcome from a large pool of feasible solutions within defined constraints. Thanks to its ability to efficiently navigate this search space, Bayesian Optimization (BO) has emerged as a go-to solution to tackle these problems [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], especially in cases where running the objective function is expensive or time-consuming. In the typical BO setup [11], a surrogate model - often a Gaussian Process (GP) regression - is leveraged to estimate the target response function from given input data. An acquisition function is then used to suggest strategic new test points based on the uncertainty level of this model. If selected carefully, this sampling strategy allows BO to explore the parameter space to uncover promising regions with high uncertainty or exploit known favorable regions to refine the search toward the global optimum.