Systematic Prompt Optimization and Testing
Deliberate Academy Editorial Team
Reviewed for accuracy and professional relevance
You're 7 lessons in — don't lose your progress.
Sign up free to save where you are and earn a verified certificate when you pass.
- Build a fixed test set of realistic inputs for a recurring prompt, rather than judging prompt quality from one or two ad hoc trials
- Define explicit scoring criteria for evaluating prompt output, replacing subjective "this looks good" judgments with a repeatable rubric
- Run regression tests when revising a prompt or after a model update, catching cases where a change improves one scenario while quietly breaking another
- Identify test-set overfitting — a prompt tuned to perform well only on the specific cases you tested — and apply the practice that prevents it
Most professionals iterate on prompts by trial and feel: try a version, glance at the output, decide it looks better or worse, adjust, repeat. This works reasonably well for low-stakes, one-off tasks and breaks down for recurring prompts that matter — you cannot reliably tell whether a revision genuinely improved a prompt, or just happened to produce a better result on the one example you happened to test it against. Systematic prompt optimization replaces vibes-based iteration with a fixed test set, explicit scoring criteria, and regression checks — the same discipline software teams apply to testing code, applied to testing prompts.
Building a Real Test Set
A test set for a recurring prompt is a fixed collection of realistic, varied inputs the prompt needs to handle well — not just the one example you happened to have on hand when you wrote the prompt. For a contract summarization prompt, a real test set includes short and long contracts, contracts with unusual clauses, contracts in different formats, and at least one edge case that has caused a problem before. The test set should be fixed and reused across every revision, so that when you change the prompt, you can compare its performance against the exact same inputs each time rather than comparing performance on different examples that happen to make the new version look better.
Purpose-built tools like Promptfoo, OpenAI's Evals framework, and LangSmith's testing features let you run a fixed test set against a prompt automatically and track results over time, which is worth adopting once a prompt is important enough to justify the setup. But the underlying discipline — a fixed, realistic test set reused across revisions — works just as well done manually in a spreadsheet for prompts that do not justify dedicated tooling.
Scoring Criteria: Replacing "Looks Good" with a Rubric
A test set is only useful if you evaluate outputs against explicit criteria rather than an overall impression. Define, before you start testing, the specific properties a good output must have: does it include every required section, does it correctly extract every key fact from the input, does it stay within a required length, does it avoid a specific known failure the prompt has produced before. Score each test case against each criterion individually — pass or fail, or a numeric scale — rather than giving the whole output a single holistic "good" or "bad" judgment, which tends to be dominated by whichever quality is most visually obvious (usually fluency) rather than the qualities that actually matter for the task.
A Fifteen-Case Test Set for Contract Summarization
Context
A legal operations manager had built a prompt to generate first-pass summaries of vendor contracts for the legal team's initial review queue. Informal testing on two or three contracts suggested it worked well, but paralegals using it reported inconsistent quality — sometimes missing a termination clause, sometimes summarizing an unusual indemnification structure incorrectly.
Action
She built a fixed test set of 15 real, anonymized contracts spanning short and long agreements, three unusual clause structures that had previously caused confusion, and two contracts in a non-standard format. She defined four scoring criteria applied to every test case: correctly identifies the termination clause, correctly identifies payment terms, flags any indemnification clause as requiring attorney review rather than summarizing its substance, and stays under 300 words. She scored the existing prompt against all 15 cases before making any changes.
Outcome
The baseline test run showed the existing prompt correctly flagged indemnification clauses in only 9 of 15 cases — in the other 6, it summarized the indemnification language directly instead of flagging it for review, exactly the failure paralegals had been reporting anecdotally. She revised the prompt to explicitly require flagging any liability-related clause by name, then re-ran the identical 15-case test set: the revision brought correct indemnification flagging to 15 of 15 without reducing accuracy on the other three criteria, which she confirmed by checking that every case that had previously scored correctly still did.
A legal operations manager revises a prompt after observing it fails to flag indemnification clauses in some contracts. What must she do before adopting the revision as the new default, according to systematic testing practice?
Select one answer.
Test-Set Overfitting: The Failure That Looks Like Success
A prompt can be iteratively refined until it scores perfectly against your specific test set while still performing poorly on real inputs that differ from anything in that set — this is test-set overfitting, and it happens when the test set is too narrow or too similar to the cases you originally noticed a problem with. A prompt tuned exclusively against six contracts from one client's template, for example, may score perfectly on that test set while still failing on a seventh contract from a different client with a different structure.
Guard against overfitting by keeping your test set genuinely varied and by periodically adding new real cases you encounter after deployment, especially cases that surprise you. A prompt that has not been tested against a new case in six months is not necessarily still working — it may simply not have been tested against anything different from what it was originally tuned on.
Regression Testing After Model Updates
The same test set that validates a prompt revision also protects against a subtler risk: the underlying AI model itself changes over time as providers release updates, and a prompt that performed reliably on one model version can behave differently on the next without any change to the prompt itself. Re-running your fixed test set periodically, and specifically after you notice a provider has released a model update, catches this kind of silent drift before it shows up as an unexplained quality complaint from whoever is relying on the prompt's output.
This lesson insists on scoring each test case against each criterion individually instead of giving the whole output one overall quality judgment. What is its stated reason?
Select one answer.
Exercise
Your Task
Choose a recurring prompt you rely on for real work. Build a fixed test set of at least eight realistic inputs, including at least one known difficult case. Define three to four explicit scoring criteria specific to what 'good output' means for this task. Score the current prompt against all test cases. If any case fails, revise the prompt and re-run the entire test set — not just the failing case — to confirm the fix did not degrade any previously passing case.
Success looks like
- Your test set includes genuinely varied inputs, not near-duplicates of the same scenario
- Your scoring criteria are specific and checkable — pass/fail against a defined property — rather than a single holistic quality judgment
- You re-ran the complete test set after any revision, confirming no previously-passing case regressed
Watch out for
- Building a test set of only two or three cases, which is not enough variation to catch how a prompt handles the genuine range of real inputs it will encounter
- Re-testing only the case that originally failed after a revision, missing a regression introduced on a different case that was previously passing
Hint
If you are not sure what to include in your test set, look back through your recent real usage of the prompt for the one or two cases where the output surprised you or needed the most manual correction — those are exactly the cases most likely to reveal a genuine weakness.
- Systematic prompt optimization replaces trial-and-feel iteration with a fixed, varied test set, explicit scoring criteria, and regression checks — the same discipline applied to testing software, applied to testing prompts.
- A test set must include realistic variation, including known difficult cases, and must be reused across every revision so comparisons are meaningful rather than comparing performance across different examples.
- Explicit, checkable scoring criteria prevent evaluation from being dominated by the most visually obvious quality — usually fluency — instead of the properties that actually determine whether the output is fit for purpose.
- Test-set overfitting — a prompt that scores perfectly on a narrow test set while still failing on realistic inputs outside it — is prevented by keeping the test set genuinely varied and periodically adding new real cases encountered after deployment.
- Purpose-built tools such as Promptfoo, OpenAI Evals, and LangSmith automate fixed test-set evaluation for prompts important enough to justify the setup, but the same discipline works manually for smaller-scale needs.