Online GentleAdaBoost -- Technical Report

Boosting algorithms belong to a class of ensemble classification approaches which use weak assumptions on the learner to efficient manner to improve performance. GentleBoost is an algorithm which was first introduced as an alternative Adaboost approach which uses Newton steps rather than exact optimization on each step (see Friedman, Hastie, and Tibshirani 2000, p353). Unlike other AdaBoost variants, GentleBoost has not received as much attention as it yields empirically inferior performance compared with other Adaboost algorithms when used on a wide range of benchmark datasets. In machine learning, the ability to extend algorithms from a batch setting to an online setting is an important topic. Online approaches can operate on streams and use datasets which are too large to fit in memory. In this technical report we provide an approach to extend GentleBoost to the online setting through using line search. In addition we perform experiments to demonstrate that the algorithm is theoretically sound and has practical usecases.