Fine-Tuning, Model Selection, and Build vs. Host

The LLM ecosystem in mid-2026 offers engineers more model choices and infrastructure options than at any previous point. There are capable open-weight models that can be self-hosted, frontier models available via managed API, specialised fine-tuned models for specific domains, and efficient smaller models that deliver acceptable quality at a fraction of the cost of the largest models.

More options mean more decisions: when to fine-tune vs. prompt, which model to use for which task, and whether to host your own inference rather than call a provider API. This lesson provides the decision framework for each of these choices — not prescriptive answers, but the criteria and tradeoffs that produce the right answer for a specific set of constraints.

When Fine-Tuning Is Justified

Fine-tuning is commonly over-prescribed. The correct sequence is: try prompting first, add RAG if prompting alone is insufficient, exhaust prompt engineering and retrieval improvements, and reach for fine-tuning only when those approaches have hit their ceiling.

Fine-tuning is justified when one of the following conditions applies:

Prompting and RAG cannot achieve the required output style or format. If the task requires a very specific output format, tone, or style that is difficult to specify in a prompt — and few-shot examples in the prompt are insufficient to achieve consistent compliance — fine-tuning can internalise the format constraint at training time. This is particularly relevant for highly structured output tasks where the schema is complex and the model consistently produces variations that frustrate validation.

The knowledge cannot be efficiently injected via context. Fine-tuning is not the right mechanism for knowledge injection (RAG is better for that), but there are cases where the task requires reasoning patterns rather than facts. If the task involves applying a complex domain-specific reasoning process that would require 5,000 tokens of examples to demonstrate via few-shot prompting, fine-tuning that reasoning pattern into the model weights produces a model that applies it consistently without the context overhead.

Latency or cost requirements make shorter prompts necessary. A fine-tuned model can often achieve comparable quality to a larger model with a long prompt, using a shorter prompt. If your latency budget is 500ms and the frontier model requires a 3,000-token system prompt to achieve the required quality, a fine-tuned smaller model with a 200-token system prompt may hit both the latency and quality target simultaneously.

Consistency is more important than generality. Fine-tuned models are less general but more consistent on the narrow task they were trained on. For production tasks where consistency is the primary quality dimension — always using exactly the right terminology, always producing exactly the right output structure — fine-tuning is more reliable than prompting.

SFT, DPO, and RLHF in Practice

Supervised Fine-Tuning (SFT) trains the model on (input, desired output) pairs using next-token prediction. The model learns to produce the desired output given the input. SFT is the simplest approach and the right starting point for most production fine-tuning use cases.

Data requirement: 500 to 5,000 high-quality (input, output) pairs, where "high quality" means the outputs are the exact responses you want the model to produce — not approximately correct, but exactly the target behaviour. The data curation step is typically the bottleneck in SFT projects.

SFT is appropriate for: teaching a model a specific output format or style, adapting a model to domain-specific vocabulary and conventions, improving consistency on a narrow task, and distilling the behaviour of a larger model into a smaller one.

Direct Preference Optimisation (DPO) trains on (input, preferred output, rejected output) triplets. The model learns to produce the preferred output over the rejected output for each input. DPO does not require a separate reward model and is substantially more computationally efficient than RLHF. It has become the standard for alignment fine-tuning in 2025-2026.

Data requirement: (input, preferred, rejected) triplets, where the distinction between preferred and rejected is consistent and clearly grounded in a specific preference dimension (e.g., more concise, better citation style, more appropriate tone). DPO is particularly effective at style transfer and preference alignment — adjusting a model's behaviour on a quality dimension without changing its knowledge.

RLHF (Reinforcement Learning from Human Feedback) trains a reward model from human preference data and uses reinforcement learning to optimise the LLM against the reward model. RLHF was the original approach for post-training alignment (used to train ChatGPT). For most production fine-tuning use cases, DPO achieves comparable results with lower computational cost and simpler implementation. RLHF is still used at the pre-training and large-scale alignment stage by foundation model providers.

LoRA and QLoRA for Efficient Fine-Tuning

Full fine-tuning updates all model weights, requiring GPU VRAM proportional to the model size (a 70B parameter model in fp16 requires 140GB of VRAM — four A100 80GB GPUs at minimum). For most teams, full fine-tuning of large models is not practical.

LoRA (Low-Rank Adaptation) adds small, trainable weight matrices (adapters) to the attention layers of the frozen base model. Only the adapter weights are trained, which are 0.1-1% of the total model parameters. LoRA reduces the trainable parameter count by 100-1,000x, enabling fine-tuning of large models on a single GPU.

QLoRA (Quantised LoRA) quantises the frozen base model weights to 4-bit or 8-bit precision before applying LoRA adapters, further reducing VRAM requirements. A 7B parameter model can be fine-tuned with QLoRA on a single 24GB GPU (RTX 4090 or equivalent). A 70B parameter model requires approximately two to four 80GB A100s with QLoRA, vs. eight or more GPUs for full fine-tuning.

The quality tradeoff: LoRA and QLoRA fine-tuned models are generally within 1-5% of full fine-tune quality on narrow tasks. For most production use cases, this is an acceptable tradeoff for the dramatic reduction in compute cost. The Hugging Face PEFT library is the standard implementation for LoRA and QLoRA in Python.

Fine-Tuning Data Requirements

The common failure mode in fine-tuning projects is quantity optimism: assuming that more training examples compensate for lower quality. They do not. 200 high-quality training examples consistently outperform 2,000 noisy or inconsistent ones.

What high-quality training data looks like. Each training example should represent the exact target behaviour: the input should be representative of the actual production input distribution, and the output should be exactly what you want the model to produce — not approximately correct, not "close enough," but the precise output that meets your quality bar. Outputs that are inconsistent with each other (different formatting, different terminology, different level of detail for similar inputs) produce inconsistent models.

Data collection strategies. The three practical sources are: expert annotation (highest quality, slowest and most expensive), distillation from a larger model with human review (moderate quality, faster), and filtering production examples where you have verified correct outputs. Do not use model-generated outputs as training targets without human review — this creates an echo chamber where the model fine-tunes toward its own existing patterns rather than toward the desired behaviour.

Minimum data volume. As a practical baseline: 200 to 500 examples for simple format/style fine-tuning, 1,000 to 3,000 examples for task-specific capability improvement, 5,000+ examples for broad domain adaptation. These are starting points — evaluate after each training run against your eval set rather than targeting a specific example count.

Model Selection Criteria

Selecting a model for a production task requires evaluating five dimensions simultaneously.

Latency. Measure p50 and p95 time-to-first-token and total generation time under your expected concurrency. Do not rely on provider-published benchmarks, which are measured under ideal conditions. Self-test under realistic load.

Cost. Calculate cost per feature at your expected traffic volume, not just per-1M-tokens pricing. Output token costs typically dominate for long-form generation; input token costs dominate for short-output classification tasks with large prompts.

Capability. Evaluate on your specific task using your own evaluation dataset — not general benchmarks. The model that scores highest on MMLU may not be the best model for your specific classification, extraction, or generation task.

Context window. If your task requires long contexts (legal document analysis, long conversation history, large retrieved contexts), verify the model's effective context window — the window length at which quality does not significantly degrade. Some models advertise large context windows but show quality degradation on tasks requiring attention to information at context extremes ("lost in the middle" problem).

Deprecation risk. Providers deprecate models on 6 to 18-month timescales. A model that is perfect for your use case today may be unavailable in 12 months. Evaluate provider deprecation track records, prefer models that have been in production for at least 6 months (suggesting a longer remaining life), and plan for model migration as a recurring engineering task.

Self-Hosted vs. Hosted Models

Self-hosted inference using vLLM, Ollama, or similar runtimes gives you: full control over the model, data residency (inputs never leave your infrastructure), no per-call API costs (only infrastructure costs), and the ability to serve custom fine-tuned models. The tradeoffs are: engineering overhead (GPU infrastructure management, model loading, serving, autoscaling), upfront infrastructure cost, and capability ceiling below frontier models.

vLLM is the production inference server for self-hosted deployment. It implements PagedAttention (efficient KV cache management that enables higher throughput and concurrent request handling), continuous batching, and tensor parallelism for multi-GPU serving. For teams serving 7B to 70B parameter models on their own infrastructure, vLLM is the standard choice.

Ollama is optimised for local development and small-scale deployment. It handles model pulling, quantisation, and a REST API in a single tool. Ollama is the right tool for local development and testing with open-weight models; vLLM is the right tool for production serving at scale.

When self-hosting is the right answer. Self-hosting makes sense when: data residency requirements prevent sending inputs to a third-party API; volume is high enough that infrastructure cost is lower than per-call API costs (typically at several hundred million tokens per day); the required capability is achievable with an open-weight model; or the use case requires a custom fine-tuned model that is not available as a managed API.

Model routing by task type. A mature production system uses multiple models for different tasks: a large frontier model for high-complexity reasoning and generation, a smaller efficient model for simple classification and extraction, and potentially a self-hosted model for a specific task where a custom fine-tune has been deployed. The routing layer (covered in Lesson 5) assigns each request to the appropriate model based on task type.