Lessons Learned From Benchmarking Fast Machine Learning Algorithms

Boosted decision trees are responsible for more than half of the winning solutions in machine learning challenges hosted at Kaggle, according to KDnuggets. In addition to superior performance, these algorithms have practical appeal as they require minimal tuning. In this post, we evaluate two popular tree boosting software packages: XGBoost and LightGBM, including their GPU implementations. All our code is open-source and can be found in this repo. We will explain the algorithms behind these libraries and evaluate them across different datasets.