AI Glossary
Caching (AI)
What is Caching (AI)?
Storing the results of expensive AI computations — such as prompt embeddings or repeated inference outputs — to avoid recomputing them on identical future requests. Caching reduces inference cost and latency in production AI systems. Prompt caching is increasingly supported by major LLM API providers for frequently repeated system prompts.
Example in practice
A team whose AI application repeatedly sends the same 2,000-token system prompt with every request would enable prompt caching — paying for the prompt once rather than re-processing it on every API call, cutting costs significantly at scale.