10 Decision Trees are Better Than 1

In the previous article of this series, I reviewed decision trees and how we can use them to make predictions. However, for many real-world problems, a single decision tree is often prone to bias and overfitting. We saw this in our example from the last blog, where even after a little hyperparameter tuning, our decision tree was still wrong 35% of the time. A solution to this poor performance problem is to use an ensemble of decision trees rather than just one. The key benefit of tree ensembles is they generally have better performance than a single decision tree. While there are many ways we could combine a set of decision trees to improve performance, two popular methods are bagging and boosting.