AI Glossary
Softmax
What is Softmax?
A mathematical function applied in neural networks to convert raw output scores (logits) into a probability distribution. In language models, softmax is applied to the vocabulary at each generation step to determine token selection probabilities. It is a core component of the transformer architecture.
Example in practice
When a language model assigns an 85% probability to "revenue" and a 5% probability to "profit" as the next word in a business sentence, that probability distribution was produced by the softmax function applied to the model's output logits.
See also
Learn more
See Softmax applied in a professional context through this free course.
AI Fundamentals for Professionals →