Agent Assurance for your AI Deployment
Test how your agents actually behave across workflows, tools, and actions. Catch failures and vulnerabilities before they ship.
npm install -g @testmuai/rook@latest
Tool Calling Benchmark: How to Build One for Your Agent
Build a tool calling benchmark from your own tools: a task format, four trajectory match modes, end-state checks and a per-task release gate, in runnable code.
Published on:
Overview
A tool calling benchmark is a fixed set of tasks with known-correct tool use, run against a model or agent to measure how reliably it chooses tools, fills arguments and leaves systems in the right state. Public leaderboards such as BFCL compare models; a benchmark built from your own tools, policies and incidents decides whether your agent ships.
What Does a Tool Calling Benchmark Check?
- Tool choice: A tool calling benchmark checks whether the agent called a tool when the task needed one and picked the right tool rather than a similar one. It also checks that the agent answers directly when the correct move is to reply, ask a question or refuse.
- Argument values: A tool calling benchmark compares each argument with the expected value after normalizing case, whitespace and number formats. Schema validity is not enough, because a schema-valid call can still refund the wrong order.
- Trajectory match: Trajectory evaluation compares the ordered list of tool calls with a reference under one of four modes: strict, unordered, superset or subset. Superset and subset each miss a different failure, so each one is paired with forbidden-tool rules.
- End state: An end-state check reads the system with a separate read-only query after the run. It catches a duplicated refund that a superset trajectory match passes, because refund_total reads 80 instead of 40.
- Repeated runs: A release gate runs each task several times and requires every high-risk task to pass repeatedly. A suite averaging 0.825 on single runs can still have three of five tasks below a pass^4 gate of 0.9.
- Agent Assurance: TestMu AI's Agent Assurance derives scenarios from an agent's code or spec and checks its tool calls against the tools the agent declares. Criteria it could not check are reported as Unable to Verify, outside the pass rate.
What is the best tool calling benchmark?
There is no single best one. Each public benchmark compares models on its own tasks, and none of them tests your own tools, so use them to shortlist a model and your own benchmark to decide a release.
- BFCL V4: The Berkeley Function Calling Leaderboard compares models on serial, parallel, multi-turn and agentic function calls, checking single calls by their syntax tree and multi-turn tasks by the state they leave. Tests your own tools: No.
- τ-bench: τ-bench puts an agent with domain API tools and policy guidelines in a conversation with a simulated user, then compares the final database state with a goal state. Simulated user: Yes.
- τ²-bench: τ²-bench adds dual control, where the simulated user also uses tools in a shared environment, and a telecom domain. User has tools: Yes.
- TRAJECT-Bench: TRAJECT-Bench scores tool selection, argument filling and call order on production-style APIs, at trajectory level as well as final accuracy. Multi-step trajectories: Yes.
- LiveMCP-101: LiveMCP-101 has 101 real-world queries that need several MCP tools together, and even frontier models succeed on fewer than 60% of them. MCP tools: Yes.
A support agent is asked to refund a late order. It looks up the order, calls issue_refund with the right order ID and amount, and then calls it again. Both calls are well-formed, the reply says the refund is done, and the customer is refunded twice.
No public leaderboard would catch that, because none of them has your issue_refund tool or your rule that a refund runs once. A tool calling benchmark built from your own tools does, and the scripts in this guide are plain Python with only the standard library, shown with the output they printed on Python 3.14.6.
What Is a Tool Calling Benchmark?
A tool calling benchmark is a set of tasks, each paired with the tool use a correct agent must produce and a rule that scores what the agent actually did. A function calling benchmark is the same thing under another name: Claude's tool use documentation says tool use is also called function calling, and OpenAI's API calls it function calling.
- Public benchmarks - the same tasks and scoring rule for every model, so leaderboard scores compare models with each other. Use them to shortlist a model.
- Your own benchmark - tasks built from your tool schemas, policy limits and past incidents, run against your agent in staging. Use it to decide whether a release ships.
Agent Assurance, from TestMu AI, generates suites of the second kind from an agent's code; its section near the end lists what it automates and what stays your job.
What Do Public Tool Calling Benchmarks Measure?
Each public benchmark gives every model the same tasks and the same scoring rule, and each one leaves out your tools. These five cover single calls, conversations with a simulated user, long trajectories and MCP servers.
| Benchmark | What it tests | How it scores | What it cannot see in your agent |
|---|---|---|---|
| BFCL V4 (leaderboard last updated April 12, 2026) | Serial and parallel function calls across programming languages, multi-turn interactions and agentic tasks | Abstract syntax tree (AST) checks of single-turn calls, state and response checks for multi-turn tasks; overall accuracy is the unweighted average of the sub-categories | Your schemas, your policy limits, and what a call changed |
| τ-bench (2024) | An agent with domain API tools and policy guidelines talking to a simulated user | Database state at the end of the conversation against the annotated goal state, plus pass^k over repeated trials | Your domain rules; it covers only its own domains |
| τ²-bench (2025) | Dual control: the user also uses tools in a shared, changing environment; adds a telecom domain | Task success, with errors split into reasoning versus communication and coordination | Your users' real tools and channels |
| TRAJECT-Bench (2025) | Tool selection, argument filling and ordering on production-style APIs, in parallel calls and dependent chains | Trajectory-level diagnostics as well as final accuracy | Side effects in your own systems |
| LiveMCP-101 (2025) | 101 real-world queries that need several MCP tools together | A reference agent runs a validated plan in parallel to produce real-time reference outputs | Your MCP servers, permissions and limits |
The papers agree on where models break:
- Long horizons - the authors of the BFCL paper (ICML 2025) write that memory, dynamic decision-making and long-horizon reasoning remain open challenges.
- Shared control - τ²-bench reports significant performance drops when agents move from settings with no active user to dual control.
- Similar tools - TRAJECT-Bench finds similar-tool confusion and parameter-blind selection, with the hardest step between short and mid-length trajectories.
- MCP tasks - LiveMCP-101 reports that even frontier models succeed on fewer than 60% of its tasks.
Which public scores to trust, how benchmarks saturate and leak, and how to run one with an open harness are covered in LLM benchmarks.
Why Don't Leaderboard Scores Predict Your Agent's Tool Calls?
A leaderboard score describes a model working on someone else's tools, and your agent works on yours, with its own schemas, rules and side effects:
- Your schemas - your tool names, parameter names, enums and descriptions appear in no public benchmark, and an ambiguous description causes wrong tool choices that no leaderboard sees.
- Your policies - a rule such as "refunds over $200 need approval" exists only in your system prompt and your code, so only your benchmark can check it.
- Side effects - an AST check confirms that a call was well-formed, not that the system ended in the right state; the same correct call made twice refunds a customer twice.
- Repetition - an agent built on a language model can take a different path on each run of the same task, which is why τ-bench measures reliability over multiple trials. One passing run is weak evidence. Measuring AI reliability across repeated, rephrased and fault-injected runs turns that into a release gate.
What Goes Into a Tool Calling Benchmark Task?
Each task is a small record kept in version control next to the agent's code. This is the refund task that the scorer later in this guide runs against:
{
"id": "refund-under-limit",
"input": "My order ORD-17 arrived late. Can I get my $40 back?",
"reference": [
{"tool": "get_order", "args": {"order_id": "ORD-17"}},
{"tool": "issue_refund",
"args": {"order_id": "ORD-17", "amount": 40}}
],
"mode": "superset",
"forbidden": ["escalate_to_human"],
"ignore_args": ["reason"],
"end_state": {"refund_total": 40, "order_status": "refunded"}
}input- the user message or conversation that starts the task.reference- the tool calls a correct agent must make, with their arguments.mode- how strictly the actual calls must match the reference: strict, unordered, superset or subset.forbidden- tools that must not be called on this task.ignore_args- arguments allowed to vary between runs, such as a free-text reason.end_state- facts about the system after the run, read directly from the test database or mock.
Write tasks from your top user intents, every policy rule with a threshold, every tool with write access and every past production incident. Add tasks where the right behavior is to answer, ask a question or refuse without calling anything.
If you are still choosing the tools themselves, the 7 trial checks to evaluate an agent-native tool grade the tool side, while this benchmark grades the agent that calls it. The case for building your own benchmark instead of trusting leaderboards is made in why agents need custom evals, a write-up of Haritha Sreedharan Nair's TestMu Conf 2026 session:
How Do You Validate a Single Tool Call?
Run these checks on each call, cheapest first:
- Call or no call - did the agent call a tool when the task needed one, and answer directly when it did not?
- Tool choice - is it the right tool rather than a similar one, the confusion TRAJECT-Bench measured?
- Schema validity - do the arguments match the JSON schema's types, required fields, enums and formats? The guide to agent tools shows how to contract-test a tool's schema with the arguments models send.
- Argument values - are the values right as well as well-typed? Compare them after normalizing case, whitespace and number formats, and use a model-graded check only for free text.
- Authority - may this agent make this call, with these values, for this user?
Provider features cover schema validity and nothing after it. OpenAI's function calling guide says setting strict to true makes calls reliably adhere to the function schema instead of being best effort, and Claude's tool definitions accept the same strict: true option.
A call that passes strict mode can still refund the wrong order, so checks 4 and 5 stay in your benchmark.
How Do You Evaluate an Agent Trajectory?
An agent trajectory is the ordered list of tool calls an agent made during a task, and trajectory evaluation compares that list with the task's reference. Pick one match mode per task:
| Mode | Passes when | Use it for | Watch out for |
|---|---|---|---|
| strict | Same calls, same order, nothing extra | Workflows where order is a rule, such as verifying identity before changing an account | Valid alternative paths fail |
| unordered | Same calls in any order, nothing extra | Independent lookups that can run in parallel | Order is not checked, and harmless extra reads fail |
| superset | Every reference call made; extra calls allowed | Agents that may look things up before acting | A duplicated write passes |
| subset | No call outside the reference; some may be skipped | Holding an agent to an allow-list of tools | A skipped required step passes |
Superset and subset each let a different failure through, so pair either one with forbidden-tool rules and an end-state check. Formulas for tool selection accuracy and trajectory precision and recall, with the ways each one can mislead, are in AI agent evaluation metrics.
How Do You Score Tool Calls and End State in Code?
Save the scorer as score.py and run python3 score.py. It scores the refund task against three recorded runs: a correct run with an extra policy lookup, a run that refunds twice, and a run that hands the customer to a human instead of refunding.
"""Score agent runs against one tool calling benchmark task."""
def norm(value):
"""Normalize values so ' ORD-17 ' == 'ord-17' and 40.0 == 40."""
if isinstance(value, str):
return value.strip().lower()
if isinstance(value, float) and value.is_integer():
return int(value)
if isinstance(value, dict):
return {k: norm(v) for k, v in value.items()}
if isinstance(value, list):
return [norm(v) for v in value]
return value
def same_call(ref, got, ignore=()):
if ref["tool"] != got["tool"]:
return False
def keep(args):
return {k: norm(v) for k, v in args.items() if k not in ignore}
return keep(ref["args"]) == keep(got["args"])
def match(reference, actual, mode, ignore=()):
"""Trajectory match modes: strict, unordered, superset, subset."""
if mode == "strict":
return len(reference) == len(actual) and all(
same_call(r, a, ignore) for r, a in zip(reference, actual))
left = list(actual)
found = 0
for r in reference:
hit = next((a for a in left if same_call(r, a, ignore)), None)
if hit is not None:
left.remove(hit)
found += 1
if mode == "unordered": # same calls, any order, nothing extra
return found == len(reference) and not left
if mode == "superset": # every reference call made, extras allowed
return found == len(reference)
if mode == "subset": # nothing outside the reference, may skip
return not left
raise ValueError(f"unknown mode: {mode}")
def score_run(task, run):
"""Return (passed, reasons) for one run of one task."""
reasons = []
called = [c["tool"] for c in run["calls"]]
for tool in task.get("forbidden", []):
if tool in called:
reasons.append(f"forbidden tool: {tool}")
ignore = task.get("ignore_args", ())
if not match(task["reference"], run["calls"], task["mode"], ignore):
reasons.append(f"trajectory fails {task['mode']}")
for key, want in task.get("end_state", {}).items():
got = run["state"].get(key)
if got != want:
reasons.append(f"{key}={got!r}, want {want!r}")
return (not reasons, reasons)
TASK = {
"id": "refund-under-limit",
"input": "My order ORD-17 arrived late. Can I get my $40 back?",
"reference": [
{"tool": "get_order", "args": {"order_id": "ORD-17"}},
{"tool": "issue_refund",
"args": {"order_id": "ORD-17", "amount": 40}},
],
"mode": "superset",
"forbidden": ["escalate_to_human"],
"ignore_args": ["reason"],
"end_state": {"refund_total": 40, "order_status": "refunded"},
}
RUNS = [
{"calls": [
{"tool": "get_order", "args": {"order_id": "ord-17"}},
{"tool": "get_policy", "args": {"topic": "refunds"}},
{"tool": "issue_refund", "args": {
"order_id": "ORD-17", "amount": 40.0, "reason": "late"}}],
"state": {"refund_total": 40, "order_status": "refunded"}},
{"calls": [
{"tool": "get_order", "args": {"order_id": "ORD-17"}},
{"tool": "issue_refund",
"args": {"order_id": "ORD-17", "amount": 40}},
{"tool": "issue_refund",
"args": {"order_id": "ORD-17", "amount": 40}}],
"state": {"refund_total": 80, "order_status": "refunded"}},
{"calls": [
{"tool": "get_order", "args": {"order_id": "ORD-17"}},
{"tool": "escalate_to_human", "args": {"queue": "billing"}}],
"state": {"refund_total": 0, "order_status": "open"}},
]
if __name__ == "__main__":
passed = 0
for i, run in enumerate(RUNS, 1):
ok, why = score_run(TASK, run)
passed += ok
print(f"run {i}: {'PASS' if ok else 'FAIL'}")
for reason in why:
print(f" - {reason}")
print(f"{TASK['id']}: {passed}/{len(RUNS)} runs passed")Output on Python 3.14.6:
$ python3 score.py
run 1: PASS
run 2: FAIL
- refund_total=80, want 40
run 3: FAIL
- forbidden tool: escalate_to_human
- trajectory fails superset
- refund_total=0, want 40
- order_status='open', want 'refunded'
refund-under-limit: 1/3 runs passed- Run 1 - passes although the agent added a policy lookup and wrote the order ID in lower case, because superset allows extra reads and the scorer normalizes values before comparing.
- Run 2 - passes the trajectory check and fails on state: two
issue_refundcalls leftrefund_totalat 80. - Run 3 - fails on every check: it called the forbidden
escalate_to_human, never calledissue_refund, and left the order open.
In a real harness, fill each run's state with a separate read-only query after the run ends. Never fill it from the agent's own summary of what it did, because that summary is the thing under test.
How Many Runs per Task Should a Release Gate Use?
Enough runs to see whether a pass repeats. pass^k, the metric τ-bench introduced, is the chance that k runs of the same task all succeed; how to calculate it and how to choose k are covered in pass@k vs pass^k.
The gate below applies it task by task to illustrative counts from five tasks run eight times each. Write-capable tasks must reach a pass^4 of 0.9, and read-only tasks a single-run rate of 0.9:
"""Per-task release gate for a tool calling benchmark."""
from math import comb
RUNS = 8 # runs per task
K = 4 # consecutive successes the gate asks for
GATE = 0.9 # minimum pass^k for a task with write access
# task id: (passes out of RUNS, has write access)
RESULTS = {
"refund-under-limit": (8, True),
"refund-over-limit": (7, True),
"update-address": (6, True),
"order-status": (8, False),
"cancel-order": (4, True),
}
def pass_hat_k(c, n, k):
"""Chance that k runs drawn from n runs with c passes all pass."""
return comb(c, k) / comb(n, k)
if __name__ == "__main__":
print(f"{'task':<20}{'pass^1':>8}{'pass^4':>8} gate")
p1_all, blocked = [], []
for task, (c, writes) in RESULTS.items():
p1, pk = c / RUNS, pass_hat_k(c, RUNS, K)
p1_all.append(p1)
ok = pk >= GATE if writes else p1 >= GATE
if not ok:
blocked.append(task)
print(f"{task:<20}{p1:>8.3f}{pk:>8.3f} "
f"{'ok' if ok else 'BLOCK'}")
print(f"mean pass^1 = {sum(p1_all) / len(p1_all):.3f}")
print(f"tasks blocking release: {len(blocked)}")
for task in blocked:
print(f" - {task}")$ python3 gate.py
task pass^1 pass^4 gate
refund-under-limit 1.000 1.000 ok
refund-over-limit 0.875 0.500 BLOCK
update-address 0.750 0.214 BLOCK
order-status 1.000 1.000 ok
cancel-order 0.500 0.014 BLOCK
mean pass^1 = 0.825
tasks blocking release: 3
- refund-over-limit
- update-address
- cancel-order- The mean hides the blockers - a single-run average of 0.825 looks close to shippable, yet three tasks fail the gate, and
cancel-orderpasses four runs in a row only 1.4% of the time. - Gate per task - a release passes only when every high-risk task clears its own threshold, so one inconsistent write tool cannot hide behind easy read-only tasks.
- Starting run counts - 8 runs for tasks with write access, at least 3 for read-only tasks, and every past incident added as a task with its own runs. Raise k for tasks that repeat often in production.
Tasks that call tools across several turns also need conversation-level checks, which multi-turn conversations covers.
Where Does Agent Assurance Fit in a Tool Calling Benchmark?
Writing the tasks, mocks and state checks by hand is the slow part of a benchmark like this one. TestMu AI's Agent Assurance is built to test how your agents actually behave across workflows, tools, and actions, and to catch failures and vulnerabilities before they ship. For a tool calling benchmark, that covers these parts:
- Scenarios from your sources - it derives functional and adversarial scenarios from the agent's code, or from its PRD or spec when there is no code to read, so you do not hand-write each task record.
- The real tool surface - discovery reads manifests, prompts, tool tables and MCP server configs, and connects to an MCP server to ask which tools it really has instead of trusting the config.
- Tool calls against declared tools - the calls the agent made are checked against the tools it declares, including "must not call" rules, so the expectation comes from the agent rather than a list written for each task.
- Read-only state checks - a judge can confirm that a record changed through a read-only query tool you approve; without one, the criterion is Unable to Verify or the scenario is skipped, never a pass.
- Three verdicts - each criterion is Pass, Fail or Unable to Verify, and Unable to Verify stays out of the pass rate in both directions.
- Adversarial by default - prompt injection, jailbreak, data exfiltration, PII leakage and policy violation scenarios are generated as a class, not added by hand.
- Run-over-run change - a second run shows the pass-rate change and which scenarios newly fail or were fixed.
The command line is Rook CLI. Installing it through npm needs Node.js 20 or newer:
npm install -g @testmuai/rook@latest
rook --versionIn CI, gate on the run's JSON rather than the exit code: a finished run exits 0 whether its scenarios passed or failed. The Agent Assurance CI/CD guide has recipes for GitHub Actions, Jenkins and Argo CD whose gate parses the JSON from rook run --json and fails closed on any failed, unverifiable or missing result.
Keep score.py, or an eval library, for fast unit-level checks on every change, and use Agent Assurance runs against staging as the release evidence. If your agent talks rather than acts, through chat, voice, phone, video or image, TestMu AI's Agent Testing is the product built for it.
Write Your First Ten Benchmark Tasks
Write ten tasks in the JSON format above from your write-capable tools and your last production incidents, run each one eight times through score.py, and hold the release on gate.py's per-task result.
To have scenarios generated from the agent's code and graded against its tool calls and records, start with the Agent Assurance quickstart.
npm install -g @testmuai/rook@latest, start a free trial, or for a full walkthrough of the product.Author
Anubhav Singhmaar is an AI Product Manager at TestMu AI driving Kane CLI, the command-line tool that brings browser automation to the terminal, turning natural-language flows into runs in a real Chrome browser that return pass or fail with shareable proof. He owns the roadmap and prioritization and works with engineering to ship developer-facing features. Before TestMu AI, he spent over four years at Sprinklr owning enterprise voice AI across APAC and EMEA. A mechanical engineer turned product manager, he grounds guidance in real QA workflows.
Reviewer
Samyak Goyal is a Senior Member of Technical Staff at TestMu AI engineering Kane CLI, the command-line tool that runs browser automation from the terminal, where a flow described in natural language executes in a real Chrome browser and returns pass or fail with shareable proof. He is a backend engineer with 4+ years of experience, previously an SDE at Innovaccer, where he built APIs, introduced Kafka, and cut deployment from weeks to hours. Samyak also builds multi-agent systems, skill-orchestration frameworks, and a personal copilot that indexes 200+ microservice repositories.
Tool Calling Benchmark FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests



















