Cheat-Sheet: Decision Trees Terminology

Now, that we know the basic building blocks of a decision tree, we need to know how to grow one. Creating a decision tree describes the process of dividing the input space into several distinct, non-overlapping sub-spaces. In order to divide the input space, we have to test all features and threshold values to find the optimal split that minimizes our cost function. Once we obtain the best split, we can continue to grow our tree recursively. The process is termed recursive since each sub-space may be split an indefinite number of times until a stopping criterion (e.g.