Stochastic Gradient Descent (SGD)

Stochastic Gradient Descent (SGD) is an optimization algorithm used in machine learning to minimize a function by iteratively moving towards the minimum value of the function. It's an approximation of gradient descent, where updates to the model parameters are made using a subset of the data rather than the full dataset.

How SGD Works

In SGD, the model parameters are updated for each training example or a small batch of training examples. This frequent updating with a limited amount of data introduces randomness in the optimization path, which can help escape local minima and often leads to faster convergence on large datasets.