Optimizing Bayesian acquisition functions in Gaussian Processes

Bayesian optimization is a popular optimization technique for optimizing a black box function especially with high dimensions. For a known objective functions, various optimization functions are readily available to choose from. For a black box function, since the true nature of the objective function is unknown, many available optimization techniques including Gradient Descent cannot be applied. For a black box function, various other optimization techniques are available such as Grid Search and Random Search, however, both of these techniques are extremely inefficient and time consuming specially if the objective function is costly to execute. Instead, Bayesian optimization tries to find the global optimum by using a surrogate function to evaluate the real objective function, thus, making the computation much efficient with respect to time or money.