Decision Boundary

A decision boundary is a hypersurface that partitions the underlying vector space into separate regions, each region corresponding to a particular class. In the context of machine learning, a decision boundary is a surface that separates instances of different class labels. It is a crucial aspect of many models in supervised learning, including logistic regression, support vector machines, neural networks, etc.

How it works

For example, in a binary classification problem using two features, the decision boundary can be a straight line that separates the data into two regions. Any point lying on one side of the line is predicted as belonging to class A, and any point on the other side is predicted as belonging to class B. The position of the decision boundary is learned by the algorithm based on the training data.

More complex decision boundaries can be nonlinear, depending on the model and the nature of the features. For instance, in a classification problem with two features but with a more complex relationship, the decision boundary could be a curve (like a circle, a parabola, etc.) or even a more complex hypersurface in higher dimensions.

The goal in training these models is to find the optimal position and shape of the decision boundary that best separates the classes in the feature space, minimizing the classification error.