Guide To Ensemble Methods: Bagging vs Boosting

Building a highly accurate prediction model is certainly a difficult task. Noise – Irreducible error i.e. the part of target value which the model is not able to predict / explain. As you know it is impossible to reduce the noise, hence the term irreducible error, we shift our focus on reducing Bias and Variance. So, Ensemble learning methods bring up the technique to reduce the Bias and Variance of the model by using multiple models together (hence the term Ensemble), in order to achieve better predictive performance, instead of a single model for prediction. There are a number of Ensemble methods, in this article I will be discussing about the two widely used Ensemble methods that are Bagging and Boosting. When we use different / single learning algorithm, multiple times for prediction.