What is Gradient Boosting and How is it different from AdaBoost?

Ensemble methods is a machine learning technique that combines several base models in order to produce one optimal predictive model. There are various ensemble methods such as stacking, blending, bagging, and boosting. Gradient Boosting, as the name suggests is a boosting method. Boosting is loosely-defined as a strategy that combines multiple simple models into a single composite model. With the introduction of more simple models, the overall model becomes a stronger predictor.