Gradient Boosting Machine: A Survey

Proposed by Freund and Schapire ( 1997), boosting is a general issue of constructing an extremely accurate prediction with numerous roughly accurate pred ictions. Addressed by Friedman ( 2001, 2002) and Natekin and Knoll ( 2013), the Gradient Boosting Machines (GBM) seeks to build predictive models through back-fittings and no n-parametric regressions. Instead of building a single model, the GBM starts by generatin g an initial model and constantly fits new models through loss function minimization to prod uce the most precise model ( Natekin and Knoll, 2013). This survey concentrates on the mathematical derivations of the gradient boosting algorithms. In Section 2, we analyze the optimization methods for par ametric and nonparametric models. Section 3 covers the definitions of different typ es of loss functions. In Section 4, we present different types of boosting algorithms, while in Section 5, we explore the combination of boosting algorithms and ranking algorithms to ran k the real-world data.