AI Glossary
Knowledge Distillation
What is Knowledge Distillation?
The process of training a smaller model to mimic the outputs of a larger model, often used interchangeably with 'model distillation'. Knowledge distillation enables organisations to deploy compact, fast models that retain much of the larger model's capability at a fraction of the inference cost.
Example in practice
A company that trains a small fast model to match the outputs of an expensive frontier model for their specific use case is applying knowledge distillation — gaining most of the quality at a fraction of the API cost.
Learn more
See Knowledge Distillation applied in a professional context through this free course.
AI Fundamentals for Professionals →