AI Glossary
Gradient Descent
What is Gradient Descent?
The optimisation algorithm used to train neural networks by iteratively adjusting model weights in the direction that reduces the loss function. Gradient descent comes in several variants — batch, stochastic, and mini-batch — each with different computational trade-offs. Understanding it contextually helps professionals evaluate why AI training is compute-intensive.
Example in practice
When an AI company runs training for weeks on thousands of GPUs, the vast majority of that compute is spent on gradient descent iterations — incrementally improving model weights across trillions of training examples.
Learn more
See Gradient Descent applied in a professional context through this free course.
AI Fundamentals for Professionals →