AI Glossary
Distillation (Model)
What is Distillation (Model)?
A technique in which a smaller 'student' model is trained to replicate the behavior of a larger 'teacher' model. Distillation produces compact, efficient models that retain much of the teacher model's capability at a fraction of the compute cost. Many smaller open-source models are produced via distillation from larger frontier models.
Example in practice
A startup building a mobile AI feature would use a distilled version of a frontier model — getting close-to-frontier quality in a model small enough to run on a device without cloud API calls.
Learn more
See Distillation (Model) applied in a professional context through this free course.
AI Fundamentals for Professionals →