Gradient Boosting explained by Alex Rogozhnikov

Gradient boosting (GB) is a machine learning algorithm developed in the late '90s that is still very popular. It produces state-of-the-art results for many commercial (and academic) applications. This page explains how the gradient boosting algorithm works using several interactive visualizations. We take a 2-dimensional regression problem and investigate how a tree is able to reconstruct the function \( y f(\vx) f(x_1, x_2) \). Play with the tree depth, then look at the tree-building process from above!