Analysis of a Random Forests Model

In a series of papers and technical reports, Breiman [9, 10, 11, 12] demonstrated that substantial gains in classification and regression accuracy can be achieved by using ensembles of trees, where each tree in the ensemble is grown in accordance with a random parameter. Final predictions are obtained by aggregating over the ensemble. As the base constituents of the ensemble are tree-structured predictors, and since each of these trees is constructed using an injection of randomness, these procedures are called "random forests". Breiman's ideas were decisively influenced by the early work of Amit and Geman [3] on geometric feature selection, the random subspace method of Ho [27] and the random split selection approach of Dietterich [21]. As highlighted by various empirical studies (see [11, 36, 20, 24, 25] for instance), random forests have emerged as serious competitors to state-of-the-art methods such as boosting (Freund [22]) and support vector machines (Shawe-Taylor and Cristianini [35]).