Tuning XGBoost Hyperparameters with Scikit Optimize

Before we get into code and muddy our hands, let us hold here for a minute and ask ourselves- if we were a computer and had been given the same problem, how would we do it? I'll assume we have only two hyperparameter values in this situation because it makes it easier to visualize. The first thing I am going to do is built a model with any random values for our two hyperparameters and see how my model performed. The next thing I would do is increase one parameter and keep one stationary just to see how my model performance responds to an increase in one of these parameters. If my model performance increases, that means I am moving my hyperparameter in the right direction.