Introduction to Python Ensembles

Ensembles have rapidly become one of the hottest and most popular methods in applied machine learning. Virtually every winning Kaggle solution features them, and many data science pipelines have ensembles in them. Put simply, ensembles combine predictions from different models to generate a final prediction, and the more models we include the better it performs. Better still, because ensembles combine baseline predictions, they perform at least as well as the best baseline model. Ensembles give us a performance boost almost for free! An input array $X$ is fed through two preprocessing pipelines and then to a set of base learners $f {(i)}$. The ensemble combines all base learner predictions into a final prediction array $P$. In this post, we'll take you through the basics of ensembles -- what they are and why they work so well -- and provide a hands-on tutorial for building basic ensembles. To illustrate how ensembles work, we'll use a data set on U.S. political contributions.