Decision Tree

A Decision Tree is a supervised machine learning algorithm used for classification and regression tasks. It represents a hierarchical model of decisions and their possible consequences, mapping observations about an item to conclusions about the item's target value. In simple terms, it is a decision support tool that uses a tree-like graph or model of decisions and their possible results, including chance event outcomes, resource costs, and utility.

How Decision Tree works

A decision tree is built top-down from a root node and involves partitioning the data into subsets that contain instances with similar values (homogenous). Internal node represents a feature in our dataset, each branch represents a decision rule, and each leaf node represents the outcome. The topmost node in a decision tree is known as the root node, it learns to partition based on the attribute value.

The process starts with the selection of the best attribute from the input data using Attribute Selection Measures(ASM), such as Information Gain, Gain ratio, and Gini Index to decide which feature is most helpful in classifying the instances. The attribute with the highest ASM is chosen as a decision node and divides the dataset into smaller subsets.

This process is recursively repeated for each derived subset in a method called recursive partitioning. It continues until either the subset at a node has the same value for the target variable, or the subset is not divisible any further. The end nodes of branches that meet these termination conditions are called leaf nodes or terminal nodes, which hold the value to be predicted.

In order to classify a new object from an input vector, start from the root node, and successfully run through it through the tree following the corresponding branches until reaching a leaf node. The leaf node provides the classification of the object.