Evaluating LLM Relevancy with DeepEval [Testμ 2026]

Sounds Right, Is Right

Ask a model a question and you get an answer that is fluent, confident and well written. Sounding right and being right are separate properties, and the gap between them is where the damage happens.

Relevancy is the deceptive failure mode because nothing about the output looks wrong. A hallucination dressed as helpfulness passes every human glance in a hurry.

The practical question the session answers is how to measure that gap in a form you can actually run in a pipeline.

The Assertion Problem

Traditional testing rests on deterministic assertions. You call a method, or hit an API, or read a value from the UI, and you assert equality against a string or a JSON structure.

LLMs break that model rather than complicate it. The same input produces different output between runs, so a string match fails a response that was perfectly correct.

Which is the whole reason a separate discipline exists. If equality cannot be the check, something has to judge meaning instead.

Pytest for LLMs

DeepEval is an open-source evaluation framework she describes as pytest for LLMs. It is a Python library you import into an existing repository, with Python as the only real prerequisite.

Instead of comparing strings, JSON or images, it hands the output to a judge model that scores it semantically, using techniques such as G-Eval. Everything lands on a scale from zero to one against a threshold you configure.

The flow she described runs in four stages, whether the application under test is a RAG pipeline, an agent or a chatbot.

• Output capture - responses become test case objects holding the input and the output.
• Metrics engine - those objects flow into the engine, which calls a judge model to score them.
• Scores with reasons - the engine returns a number, a written reason, and a pass or fail verdict.
• Reports - results are presented like any other automation report, showing what passed and failed per metric.

The Metrics Taxonomy

The metrics are organised into five categories, and she spent most time on the first because it carries the most weight when judging an agent’s response.

• RAG evaluation - faithfulness, answer relevancy, contextual relevancy, contextual precision and contextual recall, assessing the retriever and generator components.
• Agent evaluation - task completion and tool correctness, including the arguments passed, validating whether the agent finished what it was asked to do.
• Safety and compliance - toxicity, bias, PII leakage, misuse and role violation.
• Custom - your own measures, built with G-Eval.
• Conversational - for multi-turn exchanges rather than single responses.

Her framing of the RAG metrics as questions is the useful part, because each one asks something different of the same answer.

• Faithfulness - is the output grounded in the retrieved context?
• Answer relevancy - does the output address the user’s query?
• Contextual relevancy - is the retrieved context relevant to the query?
• Contextual precision - is the retrieved context focused, without noise or out-of-scope material?
• Contextual recall - does the context cover all the necessary information?

Setting Up the Project

The demo starts from an empty folder. She initialises the project with uv, which she compares to running an init command for a Node project, then adds DeepEval alongside pandas, numpy and pytest.

The judge model needs credentials. She installs Google’s generative AI package and points DeepEval at a Google AI Studio key, noting that the provider is interchangeable and the requirement is not.

Whichever model you judge with, you need an API key for it, and that key is what the cost and the speed of every run depend on.

The rest is ordinary Python hygiene: a project metadata file, a pinned Python version, then a virtual environment created, activated, and dependencies installed from the requirements file.

Writing the First Test

Her subject is an order sub-agent responsible for checking a customer’s order, tested from a Python file in a test folder.

The imports are pytest plus DeepEval’s test case type and whichever metrics you intend to use, and she picks answer relevancy. The test itself is three moves: construct a test case with an input and an expected output, evaluate it against the metric, and assert on the resulting score.

Alongside relevancy she showed the wider set available by name, covering correctness, completeness, consistency, accuracy, precision, recall and F1 score.

Her threshold in the demo is 0.8, and her reasoning is the same non-determinism that broke assertions in the first place. The agent will answer correctly using different words and terminology each time, so the bar has to allow for that.

Running it is a single command against the test file. Cases execute in parallel, and how long that takes depends on the model and on the billing tier behind your key, with free keys being noticeably slow.

Reading the Results

The output is sectioned per metric, with a score, the threshold it was measured against, and a written reason. Her passing example scored 1.0 against a 0.7 threshold on answer relevancy, because the output addressed the prompt without including irrelevant statements.

Correctness gave its own reason on the same run: the output accurately stated that Tokyo is the capital of Japan, matched the expected output, and contained no contradictions or missing details. Faithfulness reported alignment with the retrieval context and no contradictions.

She had deliberately broken some tests, and the interesting failure is not a low score. Some rows show none where a number should be.

A score of none means the agent’s output was not in the format DeepEval had parsed for the other tests. It is a format mismatch rather than a quality failure, often because the response was too large to interpret or carried extra fields, and it is fixed by correcting the output rather than by tuning a threshold.

G-Eval and Custom Metrics

The built-in metrics cover the standard RAG measures. G-Eval is the mechanism inside DeepEval for defining your own, in the same shape as relevancy but measuring whatever your application actually cares about.

Safety and compliance testing runs the same way as part of an end-to-end execution, covering toxicity, bias, PII leakage, misuse and non-advice violation.

She also named the alternatives rather than pretending there are none. Ragas and Promptfoo are the two she is aware of, while her own teams have used DeepEval for most of their sub-agent testing.

Wiring It Into CI/CD

Her position on pipelines is that nothing special is required. Eval tests are added and their results uploaded exactly like the UI and API tests already running, on GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Buildkite or an in-house system.

That is the argument for treating eval as a quality gate rather than an experiment someone runs locally before a demo.

Evals in the Test Pyramid

The question she gets most often is where eval tests belong in the pyramid, and her answer refuses to pick a single layer.

If the goal is only to see the agent’s response, and developers have given you an API or a direct call to reach it, the eval sits comfortably as a unit test.

If the goal is end to end, meaning how the agent is integrated into the UI or how the chatbot responds in place, it becomes an integration test because two components are involved.

Which layer you put them in is a decision about what you are trying to learn, and she is explicit that they can live at more than one.

Q & A Session

The Q&A ran long, and the answers are more useful than the demo in places.

• Where has DeepEval disagreed with your human judgment, and who won?

  Monika Sharma: Human judgment won. Her example is a website chatbot asked how a product works, which replied with text, images, emoticons and a row of stars inviting feedback. A person can see that response worked. DeepEval could not properly recognise the components that were not text, and her conclusion is that where an answer involves several component types beyond text, it may not be a good fit.

• What does running these evals at scale cost, since the metrics call an LLM?

  Monika Sharma: It comes down to billing, and her own rounds ran on a free API key without much cost. Free keys cap requests per minute and per day, so staying inside those limits keeps it cheap and exceeding them does not. Scheduling four or five runs a day across a suite covering many sub-agents rather than one is where it starts to demand attention.

• How can human review scale without recreating the bottleneck evaluation frameworks are meant to remove?

  Monika Sharma: The same way it does for any automation. Ten scenarios you can check with your own eyes; a hundred, or a regression suite, you cannot, and nobody is present for every push and pull request. That is the argument for a safety net that catches changes automatically, which is what DeepEval exists to be, and she was clear it is one library among several worth exploring rather than the only option.

• For an agent that plans, uses tools and recovers from failures, should evaluation focus on each decision or on the final outcome?

  Monika Sharma: Not the final outcome alone. Her unit is the utterance, meaning what a user actually sends to an agent, and one utterance against one sub-agent proves very little. Separate sub-agents handle onboarding, billing and orders, and each needs many utterances covering multiple languages, unusual symbols and the local language a site is hosted in. The steps in between are where the evaluation has to reach.

• Should agents be allowed to modify the knowledge base they use for future decisions?

  Monika Sharma: A flat no, on security grounds. In her experience organisations constrain knowledge base updates as a matter of policy, and an agent is given read access to use with retrieval rather than permission to write.

• How do you decide the right threshold for each metric?

  Monika Sharma: There is no single number. She starts at 0.5 while establishing what the agent’s response quality actually is, then raises the bar as the agents prove stable, stop hallucinating much, and are running on current models with the context they need. The threshold tracks model stability rather than ambition.

• How easy is it to adjust the safety and compliance parameters?

  Monika Sharma: Straightforward, and like every other metric they hinge on the threshold. For anything beyond the built-in set she recommends G-Eval, which she finds considerably easier to customise than the packaged metrics.

• How should enterprises establish a minimum evidence threshold before giving an agent production autonomy?

  Monika Sharma: By volume and by stability rather than by a single passing test. On past projects they tested with roughly 40 different utterances hitting the same agent, checking relevancy, precision and completeness across all of them, and with several agents in play that breadth matters more again.

This session was part of Testμ Conf 2026, which ran across three days of sessions on agentic engineering and quality. Registrations for the next edition are already open on the Testμ Conference 2027 page.