Conservation machine learning

Ensemble techniques--wherein a model is composed of multiple (possibly) weaker models--are prevalent nowadays within the field of machine learning (ML). Well-known methods such as bagging [1], boosting [2], and stacking [3] are ML mainstays, widely (and fruitfully) deployed on a daily basis. Generally speaking, there are two types of ensemble methods, the first generating models in sequence--e.g., AdaBoost [2]--the latter in a parallel manner--e.g., random forests [4] and evolutionary algorithms [5]. AdaBoost (Adaptive Boosting) is an ML meta-algorithm that is used in conjunction with other types of learning algorithms to improve performance. The output of so-called "weak learners" is combined into a weighted sum that represents the final output of the boosted classifier.