If you are in the fields of data science or machine learning, chances are you already are doing optimization! For example, training a neural network is an optimization problem, as we want to find the set of model weights that best minimizes the loss function. Finding the set of hyper parameters that results in the best performing model is another optimization problem. Optimization algorithms come in many forms, each created to solve a particular type of problem. In particular, one type of problem commonly faced by scientists in both academia and industry is the optimization of expensive-to-evaluate black box functions.