Fine-tuning XGBoost in Python like a boss – Towards Data Science

XGBoost (or eXteme Gradient Boosting) is not to be introduced anymore, proved relevant in only too many data science competitions, is still one model that is tricky to fine-tune if you have only been starting playing with it. Because if you have big datasets, and you run a naive grid search on 5 different parameters and having for each of them 5 possible values, then you'll have 5⁵ 3,125 iterations to go. If one iteration takes 10 minutes to run, you'll have more than 21 days to wait before getting your parameters (I don't talk about Python crashing, without letting you know, and you waiting too long before realizing it). I suppose here that you made correctly your job of feature engineering first. Specifically with categorical features, since XGBoost does not take categorical features in input.