AI Glossary
Quantization
What is Quantization?
A technique that reduces the numerical precision of a model's parameters (e.g. from 32-bit to 4-bit floats) to decrease file size and memory requirements. Enables large models to run on consumer hardware with some tradeoff in output quality.
Example in practice
An engineering team running an AI assistant on a local laptop rather than a cloud server would use a quantised version of a model — accepting slight quality tradeoffs in exchange for offline capability and zero API costs.
See also