Winning Kaggle 101: Introduction to Stacking

Random Forest) • Used to ensemble a diverse group of strong learners • Involves training a second-level machine learning algorithm called a "metalearner" to learn the optimal combination of the base learners 5. History of Stacking • Leo Breiman, "Stacked Regressions" (1996) • Modified algorithm to use CV to generate level-one data • Blended Neural Networks and GLMs (separately) Stacked Generalization Stacked Regressions Super Learning • David H. Wolpert, "Stacked Generalization" (1992) • First formulation of stacking via a metalearner • Blended Neural Networks • Mark van der Laan et al., "Super Learner" (2007) • Provided the theory to prove that the Super Learner is the asymptotically optimal combination • First R implementation in 2010 6.