Multi-class Classification

Multi-class classification refers to a type of machine learning problem where an instance can belong to one of more than two classes. In other words, it is a classification task with two or more than two classes. xamples can include classifying an email into multiple categories such as 'business', 'spam', 'social', 'promotions' etc. or classifying a news article into categories like 'sports', 'politics', 'entertainment' etc.

How Multi-class Classification works

In a typical multi-class classification problem, the learning algorithm is tasked with learning from a set of features to predict one of the several possible outcomes. The prediction model maps input features to the output classes using learned weights. The model is trained using a labeled dataset where each instance is associated with a specific class.

The most common approach to solving a multi-class classification problem involves training multiple binary classifiers, each dedicated to distinguishing between a pair of classes (One-vs-One) or one class and the rest (One-vs-All). For a new instance to be classified, all binary classifiers make their predictions and the one with the highest decision function score is picked as the class for that instance.

There are also algorithms that can handle multi-class problems natively without having to break them down into binary. Examples include Decision trees, Random Forest, Naive Bayes, Nearest Neighbors, and some types of Neural Networks. In these methods, the learning algorithm adjusts the boundaries for each class in the feature space to achieve the best classification.

The performance of a multi-class classification model is often measured through metrics such as accuracy, confusion matrix, log loss, or more advanced metrics like macro-averaged F1-score that takes into consideration class imbalance.