How to Implement and Evaluate Decision Tree classifiers from scikit-learn

A Decision Tree follows a tree-like structure (hence the name) whereby a node represents a specific attribute, a branch represents a decision rule, and leaf nodes represent an outcome. We will show this structure later so you can see what we mean but you can imagine it is like one of the decision trees you used to draw in high school maths, just on a far more complicated scale. The algorithm itself works by splitting the data according to different attributes at each node while attempting to reduce a selection measure (often the Gini index). In essence, the aim of a Decision Tree classifier is to split the data according to attributes while being able to classify the data accurately into predefined groups (our target variable). For this decision tree implementation we will use the iris dataset from sklearn which is relatively simple to understand and is easy to implement.