Automate Stacking In Python: How to Boost Your Performance While Saving Time

Utilizing stacking (stacked generalizations) is a very hot topic when it comes to pushing your machine learning algorithm to new heights. For instance, most if not all winning Kaggle submissions nowadays make use of some form of stacking or a variation of it. First introduced in the 1992 paper Stacked Generalization by David Wolpert, their main purpose is to reduce the generalization error. According to Wolpert, they can be understood "as a more sophisticated version of cross-validation". While Wolpert himself noted at the time that large parts of stacked generalizations are "black art", it seems that building larger and larger stacked generalizations win over smaller stacked generalizations.