Agent Assurance for your AI Deployment

Test how your agents actually behave across workflows, tools, and actions. Catch failures and vulnerabilities before they ship.

npm install -g @testmuai/rook@latest

Start Free Trial
AIAI Testing

Tool Calling Benchmark: How to Build One for Your Agent

Build a tool calling benchmark from your own tools: a task format, four trajectory match modes, end-state checks and a per-task release gate, in runnable code.

Published on:

Overview

A tool calling benchmark is a fixed set of tasks with known-correct tool use, run against a model or agent to measure how reliably it chooses tools, fills arguments and leaves systems in the right state. Public leaderboards such as BFCL compare models; a benchmark built from your own tools, policies and incidents decides whether your agent ships.

What Does a Tool Calling Benchmark Check?

  • Tool choice: A tool calling benchmark checks whether the agent called a tool when the task needed one and picked the right tool rather than a similar one. It also checks that the agent answers directly when the correct move is to reply, ask a question or refuse.
  • Argument values: A tool calling benchmark compares each argument with the expected value after normalizing case, whitespace and number formats. Schema validity is not enough, because a schema-valid call can still refund the wrong order.
  • Trajectory match: Trajectory evaluation compares the ordered list of tool calls with a reference under one of four modes: strict, unordered, superset or subset. Superset and subset each miss a different failure, so each one is paired with forbidden-tool rules.
  • End state: An end-state check reads the system with a separate read-only query after the run. It catches a duplicated refund that a superset trajectory match passes, because refund_total reads 80 instead of 40.
  • Repeated runs: A release gate runs each task several times and requires every high-risk task to pass repeatedly. A suite averaging 0.825 on single runs can still have three of five tasks below a pass^4 gate of 0.9.
  • Agent Assurance: TestMu AI's Agent Assurance derives scenarios from an agent's code or spec and checks its tool calls against the tools the agent declares. Criteria it could not check are reported as Unable to Verify, outside the pass rate.

What is the best tool calling benchmark?

There is no single best one. Each public benchmark compares models on its own tasks, and none of them tests your own tools, so use them to shortlist a model and your own benchmark to decide a release.

  • BFCL V4: The Berkeley Function Calling Leaderboard compares models on serial, parallel, multi-turn and agentic function calls, checking single calls by their syntax tree and multi-turn tasks by the state they leave. Tests your own tools: No.
  • τ-bench: τ-bench puts an agent with domain API tools and policy guidelines in a conversation with a simulated user, then compares the final database state with a goal state. Simulated user: Yes.
  • τ²-bench: τ²-bench adds dual control, where the simulated user also uses tools in a shared environment, and a telecom domain. User has tools: Yes.
  • TRAJECT-Bench: TRAJECT-Bench scores tool selection, argument filling and call order on production-style APIs, at trajectory level as well as final accuracy. Multi-step trajectories: Yes.
  • LiveMCP-101: LiveMCP-101 has 101 real-world queries that need several MCP tools together, and even frontier models succeed on fewer than 60% of them. MCP tools: Yes.

A support agent is asked to refund a late order. It looks up the order, calls issue_refund with the right order ID and amount, and then calls it again. Both calls are well-formed, the reply says the refund is done, and the customer is refunded twice.

No public leaderboard would catch that, because none of them has your issue_refund tool or your rule that a refund runs once. A tool calling benchmark built from your own tools does, and the scripts in this guide are plain Python with only the standard library, shown with the output they printed on Python 3.14.6.

What Is a Tool Calling Benchmark?

A tool calling benchmark is a set of tasks, each paired with the tool use a correct agent must produce and a rule that scores what the agent actually did. A function calling benchmark is the same thing under another name: Claude's tool use documentation says tool use is also called function calling, and OpenAI's API calls it function calling.

  • Public benchmarks - the same tasks and scoring rule for every model, so leaderboard scores compare models with each other. Use them to shortlist a model.
  • Your own benchmark - tasks built from your tool schemas, policy limits and past incidents, run against your agent in staging. Use it to decide whether a release ships.

Agent Assurance, from TestMu AI, generates suites of the second kind from an agent's code; its section near the end lists what it automates and what stays your job.

What Do Public Tool Calling Benchmarks Measure?

Each public benchmark gives every model the same tasks and the same scoring rule, and each one leaves out your tools. These five cover single calls, conversations with a simulated user, long trajectories and MCP servers.

BenchmarkWhat it testsHow it scoresWhat it cannot see in your agent
BFCL V4 (leaderboard last updated April 12, 2026)Serial and parallel function calls across programming languages, multi-turn interactions and agentic tasksAbstract syntax tree (AST) checks of single-turn calls, state and response checks for multi-turn tasks; overall accuracy is the unweighted average of the sub-categoriesYour schemas, your policy limits, and what a call changed
τ-bench (2024)An agent with domain API tools and policy guidelines talking to a simulated userDatabase state at the end of the conversation against the annotated goal state, plus pass^k over repeated trialsYour domain rules; it covers only its own domains
τ²-bench (2025)Dual control: the user also uses tools in a shared, changing environment; adds a telecom domainTask success, with errors split into reasoning versus communication and coordinationYour users' real tools and channels
TRAJECT-Bench (2025)Tool selection, argument filling and ordering on production-style APIs, in parallel calls and dependent chainsTrajectory-level diagnostics as well as final accuracySide effects in your own systems
LiveMCP-101 (2025)101 real-world queries that need several MCP tools togetherA reference agent runs a validated plan in parallel to produce real-time reference outputsYour MCP servers, permissions and limits

The papers agree on where models break:

  • Long horizons - the authors of the BFCL paper (ICML 2025) write that memory, dynamic decision-making and long-horizon reasoning remain open challenges.
  • Shared control - τ²-bench reports significant performance drops when agents move from settings with no active user to dual control.
  • Similar tools - TRAJECT-Bench finds similar-tool confusion and parameter-blind selection, with the hardest step between short and mid-length trajectories.
  • MCP tasks - LiveMCP-101 reports that even frontier models succeed on fewer than 60% of its tasks.

Which public scores to trust, how benchmarks saturate and leak, and how to run one with an open harness are covered in LLM benchmarks.

Why Don't Leaderboard Scores Predict Your Agent's Tool Calls?

A leaderboard score describes a model working on someone else's tools, and your agent works on yours, with its own schemas, rules and side effects:

  • Your schemas - your tool names, parameter names, enums and descriptions appear in no public benchmark, and an ambiguous description causes wrong tool choices that no leaderboard sees.
  • Your policies - a rule such as "refunds over $200 need approval" exists only in your system prompt and your code, so only your benchmark can check it.
  • Side effects - an AST check confirms that a call was well-formed, not that the system ended in the right state; the same correct call made twice refunds a customer twice.
  • Repetition - an agent built on a language model can take a different path on each run of the same task, which is why τ-bench measures reliability over multiple trials. One passing run is weak evidence. Measuring AI reliability across repeated, rephrased and fault-injected runs turns that into a release gate.

What Goes Into a Tool Calling Benchmark Task?

Each task is a small record kept in version control next to the agent's code. This is the refund task that the scorer later in this guide runs against:

{
  "id": "refund-under-limit",
  "input": "My order ORD-17 arrived late. Can I get my $40 back?",
  "reference": [
    {"tool": "get_order", "args": {"order_id": "ORD-17"}},
    {"tool": "issue_refund",
     "args": {"order_id": "ORD-17", "amount": 40}}
  ],
  "mode": "superset",
  "forbidden": ["escalate_to_human"],
  "ignore_args": ["reason"],
  "end_state": {"refund_total": 40, "order_status": "refunded"}
}
  • input - the user message or conversation that starts the task.
  • reference - the tool calls a correct agent must make, with their arguments.
  • mode - how strictly the actual calls must match the reference: strict, unordered, superset or subset.
  • forbidden - tools that must not be called on this task.
  • ignore_args - arguments allowed to vary between runs, such as a free-text reason.
  • end_state - facts about the system after the run, read directly from the test database or mock.

Write tasks from your top user intents, every policy rule with a threshold, every tool with write access and every past production incident. Add tasks where the right behavior is to answer, ask a question or refuse without calling anything.

If you are still choosing the tools themselves, the 7 trial checks to evaluate an agent-native tool grade the tool side, while this benchmark grades the agent that calls it. The case for building your own benchmark instead of trusting leaderboards is made in why agents need custom evals, a write-up of Haritha Sreedharan Nair's TestMu Conf 2026 session:

Youtube thumbnail

How Do You Validate a Single Tool Call?

Run these checks on each call, cheapest first:

  • Call or no call - did the agent call a tool when the task needed one, and answer directly when it did not?
  • Tool choice - is it the right tool rather than a similar one, the confusion TRAJECT-Bench measured?
  • Schema validity - do the arguments match the JSON schema's types, required fields, enums and formats? The guide to agent tools shows how to contract-test a tool's schema with the arguments models send.
  • Argument values - are the values right as well as well-typed? Compare them after normalizing case, whitespace and number formats, and use a model-graded check only for free text.
  • Authority - may this agent make this call, with these values, for this user?

Provider features cover schema validity and nothing after it. OpenAI's function calling guide says setting strict to true makes calls reliably adhere to the function schema instead of being best effort, and Claude's tool definitions accept the same strict: true option.

A call that passes strict mode can still refund the wrong order, so checks 4 and 5 stay in your benchmark.

How Do You Evaluate an Agent Trajectory?

An agent trajectory is the ordered list of tool calls an agent made during a task, and trajectory evaluation compares that list with the task's reference. Pick one match mode per task:

ModePasses whenUse it forWatch out for
strictSame calls, same order, nothing extraWorkflows where order is a rule, such as verifying identity before changing an accountValid alternative paths fail
unorderedSame calls in any order, nothing extraIndependent lookups that can run in parallelOrder is not checked, and harmless extra reads fail
supersetEvery reference call made; extra calls allowedAgents that may look things up before actingA duplicated write passes
subsetNo call outside the reference; some may be skippedHolding an agent to an allow-list of toolsA skipped required step passes

Superset and subset each let a different failure through, so pair either one with forbidden-tool rules and an end-state check. Formulas for tool selection accuracy and trajectory precision and recall, with the ways each one can mislead, are in AI agent evaluation metrics.

How Do You Score Tool Calls and End State in Code?

Save the scorer as score.py and run python3 score.py. It scores the refund task against three recorded runs: a correct run with an extra policy lookup, a run that refunds twice, and a run that hands the customer to a human instead of refunding.

"""Score agent runs against one tool calling benchmark task."""


def norm(value):
    """Normalize values so ' ORD-17 ' == 'ord-17' and 40.0 == 40."""
    if isinstance(value, str):
        return value.strip().lower()
    if isinstance(value, float) and value.is_integer():
        return int(value)
    if isinstance(value, dict):
        return {k: norm(v) for k, v in value.items()}
    if isinstance(value, list):
        return [norm(v) for v in value]
    return value


def same_call(ref, got, ignore=()):
    if ref["tool"] != got["tool"]:
        return False
    def keep(args):
        return {k: norm(v) for k, v in args.items() if k not in ignore}
    return keep(ref["args"]) == keep(got["args"])


def match(reference, actual, mode, ignore=()):
    """Trajectory match modes: strict, unordered, superset, subset."""
    if mode == "strict":
        return len(reference) == len(actual) and all(
            same_call(r, a, ignore) for r, a in zip(reference, actual))
    left = list(actual)
    found = 0
    for r in reference:
        hit = next((a for a in left if same_call(r, a, ignore)), None)
        if hit is not None:
            left.remove(hit)
            found += 1
    if mode == "unordered":  # same calls, any order, nothing extra
        return found == len(reference) and not left
    if mode == "superset":  # every reference call made, extras allowed
        return found == len(reference)
    if mode == "subset":  # nothing outside the reference, may skip
        return not left
    raise ValueError(f"unknown mode: {mode}")


def score_run(task, run):
    """Return (passed, reasons) for one run of one task."""
    reasons = []
    called = [c["tool"] for c in run["calls"]]
    for tool in task.get("forbidden", []):
        if tool in called:
            reasons.append(f"forbidden tool: {tool}")
    ignore = task.get("ignore_args", ())
    if not match(task["reference"], run["calls"], task["mode"], ignore):
        reasons.append(f"trajectory fails {task['mode']}")
    for key, want in task.get("end_state", {}).items():
        got = run["state"].get(key)
        if got != want:
            reasons.append(f"{key}={got!r}, want {want!r}")
    return (not reasons, reasons)


TASK = {
    "id": "refund-under-limit",
    "input": "My order ORD-17 arrived late. Can I get my $40 back?",
    "reference": [
        {"tool": "get_order", "args": {"order_id": "ORD-17"}},
        {"tool": "issue_refund",
         "args": {"order_id": "ORD-17", "amount": 40}},
    ],
    "mode": "superset",
    "forbidden": ["escalate_to_human"],
    "ignore_args": ["reason"],
    "end_state": {"refund_total": 40, "order_status": "refunded"},
}

RUNS = [
    {"calls": [
        {"tool": "get_order", "args": {"order_id": "ord-17"}},
        {"tool": "get_policy", "args": {"topic": "refunds"}},
        {"tool": "issue_refund", "args": {
            "order_id": "ORD-17", "amount": 40.0, "reason": "late"}}],
     "state": {"refund_total": 40, "order_status": "refunded"}},
    {"calls": [
        {"tool": "get_order", "args": {"order_id": "ORD-17"}},
        {"tool": "issue_refund",
         "args": {"order_id": "ORD-17", "amount": 40}},
        {"tool": "issue_refund",
         "args": {"order_id": "ORD-17", "amount": 40}}],
     "state": {"refund_total": 80, "order_status": "refunded"}},
    {"calls": [
        {"tool": "get_order", "args": {"order_id": "ORD-17"}},
        {"tool": "escalate_to_human", "args": {"queue": "billing"}}],
     "state": {"refund_total": 0, "order_status": "open"}},
]

if __name__ == "__main__":
    passed = 0
    for i, run in enumerate(RUNS, 1):
        ok, why = score_run(TASK, run)
        passed += ok
        print(f"run {i}: {'PASS' if ok else 'FAIL'}")
        for reason in why:
            print(f"  - {reason}")
    print(f"{TASK['id']}: {passed}/{len(RUNS)} runs passed")

Output on Python 3.14.6:

$ python3 score.py
run 1: PASS
run 2: FAIL
  - refund_total=80, want 40
run 3: FAIL
  - forbidden tool: escalate_to_human
  - trajectory fails superset
  - refund_total=0, want 40
  - order_status='open', want 'refunded'
refund-under-limit: 1/3 runs passed
  • Run 1 - passes although the agent added a policy lookup and wrote the order ID in lower case, because superset allows extra reads and the scorer normalizes values before comparing.
  • Run 2 - passes the trajectory check and fails on state: two issue_refund calls left refund_total at 80.
  • Run 3 - fails on every check: it called the forbidden escalate_to_human, never called issue_refund, and left the order open.

In a real harness, fill each run's state with a separate read-only query after the run ends. Never fill it from the agent's own summary of what it did, because that summary is the thing under test.

How Many Runs per Task Should a Release Gate Use?

Enough runs to see whether a pass repeats. pass^k, the metric τ-bench introduced, is the chance that k runs of the same task all succeed; how to calculate it and how to choose k are covered in pass@k vs pass^k.

The gate below applies it task by task to illustrative counts from five tasks run eight times each. Write-capable tasks must reach a pass^4 of 0.9, and read-only tasks a single-run rate of 0.9:

"""Per-task release gate for a tool calling benchmark."""
from math import comb

RUNS = 8  # runs per task
K = 4  # consecutive successes the gate asks for
GATE = 0.9  # minimum pass^k for a task with write access

# task id: (passes out of RUNS, has write access)
RESULTS = {
    "refund-under-limit": (8, True),
    "refund-over-limit": (7, True),
    "update-address": (6, True),
    "order-status": (8, False),
    "cancel-order": (4, True),
}


def pass_hat_k(c, n, k):
    """Chance that k runs drawn from n runs with c passes all pass."""
    return comb(c, k) / comb(n, k)


if __name__ == "__main__":
    print(f"{'task':<20}{'pass^1':>8}{'pass^4':>8}  gate")
    p1_all, blocked = [], []
    for task, (c, writes) in RESULTS.items():
        p1, pk = c / RUNS, pass_hat_k(c, RUNS, K)
        p1_all.append(p1)
        ok = pk >= GATE if writes else p1 >= GATE
        if not ok:
            blocked.append(task)
        print(f"{task:<20}{p1:>8.3f}{pk:>8.3f}  "
              f"{'ok' if ok else 'BLOCK'}")
    print(f"mean pass^1 = {sum(p1_all) / len(p1_all):.3f}")
    print(f"tasks blocking release: {len(blocked)}")
    for task in blocked:
        print(f"  - {task}")
$ python3 gate.py
task                  pass^1  pass^4  gate
refund-under-limit     1.000   1.000  ok
refund-over-limit      0.875   0.500  BLOCK
update-address         0.750   0.214  BLOCK
order-status           1.000   1.000  ok
cancel-order           0.500   0.014  BLOCK
mean pass^1 = 0.825
tasks blocking release: 3
  - refund-over-limit
  - update-address
  - cancel-order
  • The mean hides the blockers - a single-run average of 0.825 looks close to shippable, yet three tasks fail the gate, and cancel-order passes four runs in a row only 1.4% of the time.
  • Gate per task - a release passes only when every high-risk task clears its own threshold, so one inconsistent write tool cannot hide behind easy read-only tasks.
  • Starting run counts - 8 runs for tasks with write access, at least 3 for read-only tasks, and every past incident added as a task with its own runs. Raise k for tasks that repeat often in production.

Tasks that call tools across several turns also need conversation-level checks, which multi-turn conversations covers.

Where Does Agent Assurance Fit in a Tool Calling Benchmark?

Writing the tasks, mocks and state checks by hand is the slow part of a benchmark like this one. TestMu AI's Agent Assurance is built to test how your agents actually behave across workflows, tools, and actions, and to catch failures and vulnerabilities before they ship. For a tool calling benchmark, that covers these parts:

  • Scenarios from your sources - it derives functional and adversarial scenarios from the agent's code, or from its PRD or spec when there is no code to read, so you do not hand-write each task record.
  • The real tool surface - discovery reads manifests, prompts, tool tables and MCP server configs, and connects to an MCP server to ask which tools it really has instead of trusting the config.
  • Tool calls against declared tools - the calls the agent made are checked against the tools it declares, including "must not call" rules, so the expectation comes from the agent rather than a list written for each task.
  • Read-only state checks - a judge can confirm that a record changed through a read-only query tool you approve; without one, the criterion is Unable to Verify or the scenario is skipped, never a pass.
  • Three verdicts - each criterion is Pass, Fail or Unable to Verify, and Unable to Verify stays out of the pass rate in both directions.
  • Adversarial by default - prompt injection, jailbreak, data exfiltration, PII leakage and policy violation scenarios are generated as a class, not added by hand.
  • Run-over-run change - a second run shows the pass-rate change and which scenarios newly fail or were fixed.

The command line is Rook CLI. Installing it through npm needs Node.js 20 or newer:

npm install -g @testmuai/rook@latest
rook --version

In CI, gate on the run's JSON rather than the exit code: a finished run exits 0 whether its scenarios passed or failed. The Agent Assurance CI/CD guide has recipes for GitHub Actions, Jenkins and Argo CD whose gate parses the JSON from rook run --json and fails closed on any failed, unverifiable or missing result.

Keep score.py, or an eval library, for fast unit-level checks on every change, and use Agent Assurance runs against staging as the release evidence. If your agent talks rather than acts, through chat, voice, phone, video or image, TestMu AI's Agent Testing is the product built for it.

Start a 14-day free trial of Agent Assurance by TestMu AI, with 4,000 credits, to test how your AI agents behave across workflows, tools, and actions

Write Your First Ten Benchmark Tasks

Write ten tasks in the JSON format above from your write-capable tools and your last production incidents, run each one eight times through score.py, and hold the release on gate.py's per-task result.

To have scenarios generated from the agent's code and graded against its tool calls and records, start with the Agent Assurance quickstart.

To run your first test, install Rook CLI with npm install -g @testmuai/rook@latest, start a free trial, or
for a full walkthrough of the product.

Author

...

Anubhav Singhmaar

Blogs: 49

  • Linkedin

Anubhav Singhmaar is an AI Product Manager at TestMu AI driving Kane CLI, the command-line tool that brings browser automation to the terminal, turning natural-language flows into runs in a real Chrome browser that return pass or fail with shareable proof. He owns the roadmap and prioritization and works with engineering to ship developer-facing features. Before TestMu AI, he spent over four years at Sprinklr owning enterprise voice AI across APAC and EMEA. A mechanical engineer turned product manager, he grounds guidance in real QA workflows.

Reviewer

...

Samyak Goyal

Reviewer

  • Linkedin

Samyak Goyal is a Senior Member of Technical Staff at TestMu AI engineering Kane CLI, the command-line tool that runs browser automation from the terminal, where a flow described in natural language executes in a real Chrome browser and returns pass or fail with shareable proof. He is a backend engineer with 4+ years of experience, previously an SDE at Innovaccer, where he built APIs, introduced Kafka, and cut deployment from weeks to hours. Samyak also builds multi-agent systems, skill-orchestration frameworks, and a personal copilot that indexes 200+ microservice repositories.

Add to Google preferred sources

Summarise with AI

Copied to Clipboard!
...

3000+ Browsers. One Platform.

See exactly how your site performs everywhere.

Try it free
...

Write Tests in Plain English with KaneAI

Create, debug, and evolve tests using natural language.

Try for free

Tool Calling Benchmark FAQs

Did you find this page helpful?

More Related Blogs

TestMu AI forEnterprise

Get access to solutions built on Enterprise
grade security, privacy, & compliance

  • Advanced access controls
  • Advanced data retention rules
  • Advanced Local Testing
  • Premium Support options
  • Early access to beta features
  • Private Slack Channel
  • Unlimited Manual Accessibility DevTools Tests