An Overview of Boosting Methods: CatBoost, XGBoost, AdaBoost, LightBoost, Histogram-Based Gradient…

In ensemble learning, it is aimed to train the model most successfully with multiple learning algorithms. In one of the ensemble learning, Bagging method, more than one model was applied to different subsamples of the same dataset in parallel. Boosting, which is another method and frequently used in practice, builds sequentially instead of parallelly and aims to train the algorithm as well as training the model. A weak algorithm trains the model, then it is re-organized according to the training results and it is made easier to learn. This modified model is then sent to the next algorithm and the second algorithm learns easier than the first one. This article contains different boosting methods that interpret this sequential method from different angles.