AI Glossary
Loss Function
What is Loss Function?
A mathematical function that measures the difference between a model's predictions and the correct answers during training. The loss function guides the optimisation process — the model is trained to minimise it. Language models typically use cross-entropy loss over predicted next-token probabilities.
Example in practice
When a language model assigns low probability to the correct next word, the loss function produces a high value — signalling that gradient descent should make large weight adjustments to improve the model's prediction.
Learn more
See Loss Function applied in a professional context through this free course.
AI Fundamentals for Professionals →