Implementing a Decision Tree From Scratch

Tree-based methods are simple and useful for interpretation since the underlying mechanisms are considered quite similar to human decision-making. The methods involve stratifying or segmenting the predictor space into a number of simpler regions. When making a prediction, we simply use the mean or mode of the region the new observation belongs to as a response value. Since the splitting rules to segment the predictor space can be best described by a tree-based structure, the supervised learning algorithm is called a Decision Tree. Decision trees can be used for both regression and classification tasks.