AI Glossary
Attention mechanism
What is Attention mechanism?
The mathematical mechanism inside transformer models that allows them to weigh the relevance of different parts of the input when generating each output token. The foundation of modern LLM architecture, introduced in the 2017 paper "Attention Is All You Need".
Example in practice
When a language model correctly uses a pronoun introduced thirty sentences earlier in a legal document, the attention mechanism is what allowed it to locate and reference that earlier context.
Learn more
See Attention mechanism applied in a professional context through this free course.
AI Fundamentals for Professionals →