Complete Guide to Parameter Tuning in Gradient Boosting (GBM) in Python

If you have been using GBM as a'black box' till now, may be it's time for you to open it and see, how it actually works! This article is inspired by Owen Zhang's (Chief Product Officer at DataRobot and Kaggle Rank 3) approach shared at NYC Data Science Academy. He delivered a 2 hours talk and I intend to condense it and present the most precious nuggets here. Boosting algorithms play a crucial role in dealing with bias variance trade-off. Unlike bagging algorithms, which only controls for high variance in a model, boosting controls both the aspects (bias & variance), and is considered to be more effective.