The 10 Algorithms Machine Learning Engineers Need to Know

The original ensemble method is Bayesian averaging, but more recent algorithms include error-correcting output coding, bagging, and boosting. So how do ensemble methods work and why are they superior to individual models? They average out biases: If you average a bunch of democratic-leaning polls and republican-leaning polls together, you will get an average something that isn't leaning either way. They reduce the variance: The aggregate opinion of a bunch of models is less noisy than the single opinion of one of the models. In finance, this is called diversification -- a mixed portfolio of many stocks will be much less variable than just one of the stocks alone.