Prompt Evaluation: Versioning, Scoring and Drift

What Is Prompt Evaluation?

Prompt evaluation is rerunning a fixed set of inputs against a prompt after any change, then comparing the results to the last approved baseline before the change ships. The change can be to the prompt, the model, the decoding settings, or the tools the prompt can reach.

The reason it needs its own name is that a prompt behaves like code and is managed like configuration. It determines product behaviour, it is edited under deadline pressure, and in most teams it ships with no test attached and no record of what it used to do.

Two things separate it from ordinary regression testing. The output is not deterministic, so a diff is the wrong comparison. And the system can change without anyone touching the repository, because the model underneath is someone else's deployment. Both are covered below.

Why Does a Better Prompt Make Things Worse?

Because prompt quality is not a single axis, and the advice that improves one behaviour routinely degrades another.

Commey's study is precise about the shape of this. Stronger output-contract instructions improved strict extraction for both models tested, while citation and content compliance declined under some generic-rule conditions, with the largest observed drop taking one model from 26 of 30 to 9 of 30. The instruction that tightened one capability loosened another.

The practical consequence is that "we improved the prompt" is not a reviewable claim. Improved at what, measured how, against which cases, compared to what. Without those four answers a prompt change is an unmeasured edit to production behaviour, and the review is somebody reading the new wording and agreeing it sounds better.

What Should You Version Alongside the Prompt?

Prompt versioning fails when it versions only the string. The words are one input among several, and the others move independently.

Keeping all five in the repository rather than in a vendor console is what makes a prompt change reviewable in a pull request, which is where the rest of your change control already lives.

How Do You Build the Baseline Set?

Coverage of failure modes, not volume of cases. A baseline earns its place by containing the things you would refuse to ship without.

• Start from production failures - Every incident where the model did something unacceptable becomes a case. These are the highest-value entries because they already happened once.
• Add the core paths - The handful of requests that represent what the feature is for. If these regress, the feature is broken regardless of what else improves.
• Add adversarial and edge inputs - Empty fields, very long inputs, mixed languages, and prompt-injection attempts. These are the cases a generic rewrite is most likely to disturb.
• Record the expectation, not the output - Store what must be true about a good answer rather than one blessed response. Storing an exact response makes the set brittle the first time phrasing changes harmlessly.
• Version it with the prompt - The set and the prompt move together, so a change to either is visible in the same diff.

Dozens of well-chosen cases outperform thousands of near-duplicates, because a regression only surfaces if some case exercises the behaviour that broke. Our guide to LLM evaluation covers assembling and maintaining evaluation datasets in more depth.

What We Measured About Score Stability

If a prompt gate compares scores, the first thing worth knowing is how much the score moves when nothing changes at all. We ran that on a scoring harness we use internally.

Method. A dual-model scorer grades one block of content across seven weighted dimensions and reports a composite out of 100. We ran it three times against byte-identical input with caching disabled, so every run was a fresh model call.

3 fresh runs, identical input, cache disabled

  composite score      91      91      91

  per-dimension, judge B:
    standalone citability     8       7       7
    query-matched phrasing    9      10      10
    structural precision      9      10      10
  judge A: identical on all seven dimensions, all three runs

The headline number was perfectly stable and the thing underneath it was not. One judge returned identical scores every run; the other moved on three of seven dimensions. The composite held at 91 because the movements offset each other, losing a point on one dimension and gaining one on two others.

A gate watching only the composite would have reported perfect stability across a run where three of seven measured qualities changed. If you threshold on a single aggregate number, offsetting movement is invisible to you by construction.

The second finding was an accident and is the more useful one. Our first three runs also returned identical scores, of 88 rather than 91, because the harness caches results on a hash of the input. Same input, same prompt, two different answers depending on whether the cache was consulted.

That is worth sitting with, because caching an evaluation on input hash is a common optimisation. A cached evaluation cannot detect a model update, since the input did not change and the cache does not know the model did. The optimisation that makes your eval affordable is the same one that blinds it to drift.

Scope: one harness, one content block, three runs. It is not a general measurement of judge variance, and a different scorer will behave differently. What it establishes is narrow: verify your own gate's stability before trusting a threshold on it, and check whether it caches.

How Do You Score a Prompt Change?

In two layers, cheapest first, because most regressions are structural and do not need a model to detect.

• Deterministic checks first - Valid JSON, required fields present, enum values in range, no forbidden content, response under a length limit. These are fast, free, and catch the failures that break downstream code rather than merely disappointing a reader.
• Scored checks second - Whether the answer is grounded in the supplied context, answers the question asked, and holds the required tone. These cost a model call per case, which is why they run after the cheap layer has already rejected the obviously broken.
• Compare against the baseline, not a fixed bar - The question is whether this version is worse than the approved one, not whether it clears an absolute score. Absolute thresholds drift as the set changes.
• Gate on sub-scores, not just the composite - Per the measurement above, an aggregate can hide offsetting movement. Fail the change if any individual dimension drops materially, even when the total holds.
• Keep a human on the verdict - A four-point drop may be a real quality loss or an artifact of the metric. That distinction is a product judgment, and automating it is how teams end up shipping regressions with a green check.

Where the scored layer uses a model to grade another model's output, the grader has its own biases and its own variance. Our guide to LLM-as-a-judge covers making that layer reliable enough to gate on.

What Is Prompt Drift and How Do You Catch It?

Prompt drift is behaviour changing without the prompt changing. The text is identical, the repository shows no commit, and the outputs are different, because the model underneath is a deployment you do not control.

This defeats change-triggered testing completely. A suite that runs when a prompt file changes will never run on the day a provider ships a checkpoint update, which is exactly the day you needed it.

• Run the baseline on a schedule - Nightly or weekly against the unchanged prompt. The point is to detect movement with no local cause, which only a time-triggered run can do.
• Pin the model version where you can - Where a provider exposes dated or numbered versions, pin them and treat an upgrade as a change requiring the same evaluation as a prompt edit.
• Disable evaluation caching for drift runs - A hash-keyed cache returns the old verdict for unchanged input, which is precisely the case you are trying to test.
• Sample live traffic into the set - Real inputs shift over time, so a baseline frozen at launch slowly stops describing how the feature is used.

Where Do Prompt Gates Break?

• The judge is the variable - Swapping the grading model changes scores more than rerunning the same one does. Version the judge as carefully as the prompt, or a gate can flip with nothing under test having moved.
• The baseline encodes the bug - Approving expected outputs generated by the current prompt bakes in whatever it currently gets wrong, and the suite then defends the defect.
• Cost pushes the gate off the critical path - Scored evaluations cost real money per run, so teams move them to a nightly job, and prompt changes start shipping before their evaluation completes.
• Scores become theatre - Once a number is a gate, the pressure is to move the number. A metric nobody can explain in product terms gets optimised rather than satisfied.
• Flaky gates get bypassed - A prompt suite that fails intermittently trains people to rerun until green, which is the same failure that afflicts any test suite. Our guide to quarantine tests covers isolating unreliable checks without losing the coverage.

Conclusion

Take the prompt currently running in production, write down ten inputs you would refuse to ship without handling, and record what a good answer must contain for each. That file is a baseline, and you now have something the next prompt edit can be measured against rather than argued about.

Then check two things about whatever scores it. Whether the aggregate hides movement in its parts, which ours did across three identical runs, and whether it caches, which ours also did and which would have hidden a model update entirely.

For evaluating agent and chatbot behaviour against expectations rather than exact strings, TestMu AI's Agent Testing runs those checks against conversational systems, and the getting started with Agent Testing documentation walks through defining those expectations for a first agent. Our write-up on prompt engineering for testing covers the authoring side that produces the prompts you end up gating, and LLM testing covers the wider application layer these prompts sit inside.