How Large Language Models Work — Explained Without the Maths
Deliberate Academy Editorial Team
Reviewed for accuracy and professional relevance
Enjoying the course?
Sign up free to track your progress and earn a verified certificate when you pass.
- Explain how large language models learn from training data through next-token prediction
- Describe what tokenisation is and how context window size affects model behavior
- Identify why hallucinations occur as an architectural property of LLMs rather than a bug
- Distinguish between base model training and fine-tuning, and explain why different models behave differently
- Apply the LLM strengths and weaknesses profile to evaluate whether a given task is a good candidate for AI assistance
You are in a product review with your engineering team. The lead developer says the new AI feature "uses a transformer-based LLM with a 128k context window." Your vendor says their tool "hallucinates less because of RLHF fine-tuning." A colleague forwards an article claiming that GPT-4 "understands" customer sentiment. You need to respond intelligently to all three without a computer science degree. Understanding how large language models actually work gives you exactly that capability.
From Raw Text to a Trained Model
A large language model is trained by repeatedly predicting the next token in a sequence across hundreds of billions of words, then adjusting its internal weights based on how far off the prediction was. What emerges is not a database of facts — it is a set of statistical associations that let the model generate plausible continuations of any text you give it, which is why "the AI knows that" is subtly the wrong way to describe what is happening. For the full mechanics of next-token prediction, training layers, and why the model cannot distinguish a reliable answer from a plausible-sounding one, see How AI Systems Actually Work. What this lesson adds is the layer above that mechanic: how tokenisation and context window size shape what you actually get back.
Training a frontier model like GPT-4o or Claude costs tens of millions of dollars in compute and months of time. This is why only a handful of organizations can train large models from scratch, while most businesses build on top of those models via APIs.
Understanding hallucination in a compliance context
Context
A legal operations team at a mid-sized financial services firm began using an LLM to draft regulatory summary memos. The tool produced fluent, confident output — but one memo cited a specific regulatory guidance document that did not exist.
Action
The head of legal operations introduced a mandatory verification step: every AI-drafted memo required each cited source to be checked against the regulator's published document index before distribution. The team also added a prompt instruction flagging when the model might be drawing on rare training data.
Outcome
Citation errors dropped to zero within two reporting cycles. The team attributed this to understanding next-token prediction — knowing the model generates plausible-looking references rather than retrieving verified ones prompted them to build the verification step into the workflow from the start.
Tokenisation and What the Model Actually Sees
You type words. The model processes tokens. A token is roughly a word-chunk — sometimes a full word, sometimes part of a word, sometimes punctuation. The word "unbelievable" might become "un," "believ," "able." Common words like "the" are typically single tokens.
This has practical consequences. AI models measure capacity in tokens, not words. Current frontier models offer context windows ranging from 128k to over 1 million tokens — GPT-4o and Claude 3.5 Sonnet, for instance, can process hundreds of thousands of words at once before older content starts to lose influence. Second, the model processes the entire context simultaneously using a mechanism called attention — not left to right the way you read.
The attention mechanism is what makes transformers so powerful. Attention allows the model to weigh relationships between any two tokens in the context window, regardless of distance. This is how the model can write a 5,000-word document with coherent references back to points made at the start.
Practical implication: the quality of your output often depends on what is in the context window. A prompt with rich, specific context almost always outperforms a vague prompt, because you are giving the model better statistical signals to work from.
When you get a weak AI output, the first fix is usually to improve the context you provided, not to regenerate. More specific background information, clearer constraints, and a concrete example of what good looks like will consistently improve results.
A colleague tells you that pasting a vague prompt into ChatGPT and getting a poor response means the AI tool is not good enough for your use case. What does the tokenisation and context window concept suggest instead?
Select one answer.
Why LLMs Excel at Some Tasks and Fail at Others
This architecture produces a specific profile of strengths and weaknesses that every professional should understand.
Strong performance: Writing, summarisation, translation, code generation in popular languages, format conversion, brainstorming, explaining concepts, and any task where the correct output resembles other correct outputs the model has seen. These map well to pattern completion.
Unreliable performance: Real-time information (models have training cutoffs and cannot browse the web unless given specific tools), precise arithmetic (the model generates digits statistically, not by calculating), proprietary knowledge not in the training data, and long chains of logical reasoning that must be exactly right at every step.
The hallucination problem: When you ask about something outside the training data, or where the correct answer is rare, the model does not say "I do not know." It generates what a plausible answer would look like. Vendor claims that their model "hallucinates less" are often meaningful — techniques like RLHF do reduce hallucination rates — but no current model has eliminated it. See AI Hallucinations for the five categories of professional work where this risk is highest and the three-step verification standard to apply before using any AI-generated claim.
Never use an LLM as a source of record for specific facts, citations, legal references, or numerical data without independent verification. The model cannot distinguish between something it knows accurately and something it has confidently generated incorrectly.
Fine-Tuning and Why Different Models Behave Differently
The base model trained purely on next-token prediction is powerful but raw. It might complete a customer service prompt with an angry response because angry responses appear in the training data, or produce harmful content. This is why the models you interact with have been through additional training stages.
Supervised fine-tuning trains the model on examples of good responses to prompts, teaching it to behave like a helpful assistant. Reinforcement learning from human feedback (RLHF) goes further: human raters compare pairs of model responses, indicate which is better, and the model is trained to produce more preferred responses. This is why Claude responds helpfully and declines harmful requests, while a raw base model would do neither.
Different vendors make different fine-tuning decisions, which explains why ChatGPT, Claude, and Gemini have distinct personalities, writing styles, and refusal behaviors even though all use transformer architectures. When you evaluate AI tools, you are partly evaluating fine-tuning decisions — the value judgments baked in about what a good response looks like.
Context window size, fine-tuning approach, and model scale together determine what a product is good for. A specialized smaller model fine-tuned on legal documents will often outperform a much larger general model on legal tasks. Scale is not everything.
What is the primary mechanism large language models use to predict the next word in a sequence?
Select one answer.
Exercise
Your Task
Take an AI output you have received in the past week — or generate one now by asking ChatGPT or Claude a factual question relevant to your field. Identify three specific factual claims in the response: a statistic, a named source or study, and a date or precise figure. Verify each one against an independent source and write a sentence for each recording whether it was accurate, partially accurate, or fabricated. Then write one sentence explaining what this tells you about where verification effort should go in your own professional AI workflows.
Success looks like
- You have selected claims that are specific and verifiable — named statistics, cited sources, or precise figures — not general statements
- You checked each claim against an original source, not a second AI tool or a summary page
- Your final sentence connects the verification outcome to a practical rule for your own workflow, not just a general observation about AI
Watch out for
- Selecting only claims that are obviously likely to be correct — the exercise only builds calibration if you test claims where you do not already know the answer
- Using another AI tool to verify the claims — verification requires an authoritative original source, not another model that may hallucinate the same fabrication
Hint
The most useful claims to test are specific numbers and named citations, not general conceptual statements — the model's statistical process is most likely to fabricate precise details because they are rare in training data.
- LLMs learn statistical patterns from enormous text datasets — they do not store or retrieve facts, they generate statistically plausible continuations of your input.
- Context quality drives output quality: rich, specific prompts produce better results because they give the model a narrower, more relevant probability distribution to generate from.
- The architecture produces predictable strengths in pattern-completion tasks like writing, summarisation, and code, and predictable weaknesses in real-time facts, precise arithmetic, and rare or private knowledge.
- Hallucinations occur when pattern-completion runs on territory where the training signal was weak — they are architectural, not accidental, and require mandatory human verification before professional use.
- Different models behave differently primarily because of fine-tuning decisions, not just model scale — evaluating a tool means assessing both its base capabilities and its fine-tuning choices.