What Are Agent Tools?

What Is an Agent Tool?

An agent tool is one capability, such as a function, an HTTP API, a database query or a shell command, packaged for a language model as a name, a description and an input schema. The model picks a tool and fills in its arguments, but it executes nothing: your application runs the call, or the provider does for tools it hosts.

Anthropic's tool use overview says Claude "determines when to call a tool based on the user's request and the tool's description. It then returns a structured call that your application runs (client tools) or that Anthropic runs (server tools)." The OpenAI API and the Model Context Protocol (MCP) name the same parts with their own fields:

• Anthropic client tools - user-defined tools and tools with Anthropic-defined schemas, such as bash and text_editor, run in your application, which sends back each tool_result.
• Anthropic server tools - tools such as web_search, web_fetch, code_execution and tool_search run on Anthropic's infrastructure, and the docs say you do not need to handle is_error results for them.
• MCP servers - in the MCP tools specification (revision 2026-07-28), a server exposes "model-controlled" tools that clients discover with tools/list and invoke with tools/call.
• Your agent harness - the code that receives each call, applies permissions and returns results is the agent harness, and the tools are what it dispatches.

Platforms and frameworks for building agents are a different sense of the word, compared in the roundup of agentic AI tools.

How Does an Agent Call a Tool, Step by Step?

An agent calls a tool in a loop: your application sends the conversation with the tool definitions, the model returns a call with arguments, the application checks and runs it, and the result goes back to the model, which answers or calls another tool. OpenAI's function calling guide describes this flow in five high-level steps, and the list below adds the input validation that the MCP specification requires of servers.

• Send the request with tools - the conversation goes to the model together with each tool's name, description and input schema.
• Receive the tool call - the model returns calls that each name a tool and carry JSON arguments; OpenAI's guide says a response "can include zero, one, or multiple calls" and that it is best practice to assume there are several.
• Validate the arguments - before anything runs, check the arguments against the schema and the caller's permissions; MCP servers MUST "Validate all tool inputs."
• Execute once, with a timeout - run the call under a time limit, and give a write an idempotency key so a repeat can be recognized.
• Return the result or the error - the output goes back as a tool result, or as an error result the model can read and act on.
• Let the model continue - the model answers the user or requests more calls, and the loop repeats until it stops calling tools.

For a function with no return value, such as send_email, OpenAI's guide says to return a string that indicates success or failure, so the model has a result to read even for a write.

Which Agent Tools Are Read-Only, and Which Change State?

A read-only tool returns information and changes nothing, such as get_order or a search; a tool with side effects writes, sends, pays, publishes or deletes something. The split sets the controls each tool needs, because a repeated read costs only time, while a repeated write can pay out the same refund again.

The MCP schema reference describes these traits with ToolAnnotations and says all of them "are hints" that "are not guaranteed to provide a faithful description of tool behavior." Its defaults, readOnlyHint false, destructiveHint true, idempotentHint false and openWorldHint true, treat an unannotated tool as one that may destroy data, repeat effects and reach an open world. Clients "should never make tool use decisions based on ToolAnnotations received from untrusted servers."

• Record the effect class in your own code - keep it beside each tool definition, since an annotation from a server is only a hint.
• Treat unknown tools as writes - until a test shows a tool changes nothing, give it the approval and retry rules of a write, as the spec's defaults do.
• Test the read-only claim - run the tool and confirm the system of record did not change, as the script later in this guide shows for get_order.

What Should an Agent Tool Schema Contain?

An agent tool schema is a JSON Schema object that names each argument with its type, lists the required ones, narrows values with enums, patterns and minimums, and rejects unknown fields with additionalProperties: false. Each constraint turns a class of wrong arguments into a validation error before the tool runs. The script's issue_refund definition, adapted below (wrapped to fit, with get_order left out), uses all of them but leaves the idempotency key optional:

const tools = {
  // get_order (read-only) omitted here
  issue_refund: {
    effect: 'write',
    description:
      'Refund one order. Writes one ledger row. ' +
      'Pass idempotency_key so a retried call ' +
      'returns the first result instead of paying twice.',
    inputSchema: {
      type: 'object',
      properties: {
        order_id: {
          type: 'string',
          pattern: '^ORD-\\d{4}$',
        },
        amount_cents: { type: 'integer', minimum: 1 },
        reason: {
          type: 'string',
          enum: ['damaged', 'late', 'wrong_item',
            'duplicate_charge'],
        },
        idempotency_key: { type: 'string' },
      },
      // idempotency_key is deliberately optional here:
      // that is the design flaw the retry check exposes.
      required: ['order_id', 'amount_cents', 'reason'],
      additionalProperties: false,
    },
  },
};

• Types - amount_cents must be an integer, so the string "1200" fails; Anthropic's strict tool use page says that without strict mode, Claude "might return incompatible types ("2" instead of 2) or omit required fields."
• Required fields - a call without order_id is rejected; idempotency_key is optional here on purpose, and the retry check later shows what that costs.
• Enums - reason accepts four values, so "goodwill" is rejected; OpenAI's guide says to "Use enums and object structure to prevent invalid states."
• Patterns and minimums - order_id must look like ORD-1001, and a refund of zero cents fails the minimum of 1.
• Unknown fields - additionalProperties: false rejects an extra note argument the model added; for a tool with no parameters, MCP recommends a schema that "explicitly accepts only empty objects."
• Arguments your code already knows - OpenAI's guide advises against making the model fill them: if your code already holds the order_id, define submit_refund() with no parameters and pass the ID in code.

Strict modes enforce the schema while the model generates the call:

• Anthropic - strict: true "guarantees Claude's tool inputs match your JSON Schema by constraining the model's token sampling to schema-valid outputs."
• OpenAI - strict: true requires additionalProperties: false on every object and every field in properties marked as required, with optional fields given null as a type option; a schema that misses these is rejected, and OpenAI recommends always enabling strict mode.

Both modes accept only part of JSON Schema. Anthropic's structured outputs page lists numerical constraints such as minimum as unsupported in strict tool use, and a request that sends one gets a 400 error, while the optional idempotency_key breaks OpenAI's rule that every field is marked required. The refund schema above would need adapting for either mode, and the tool should still validate its own inputs.

Neither provider says strict mode checks whether a value is correct, only that it fits the schema, so a schema-valid call can still refund the wrong order. In τ-bench, gpt-4o's "wrong argument" failures came from calls where it "usually makes the right type of tool call(s) but fills in one or more arguments incorrectly."

Missing values are a related risk: when a prompt lacks a required parameter, Anthropic's overview says Claude Opus is much more likely to ask for it, while Claude Sonnet "might also infer a reasonable value," and a schema check cannot tell an inferred order ID from a real one.

The OWASP Top 10 for Agentic Applications 2026 turns that gap into a control: validate "the intended semantics of tool calls (e.g., query type or category) rather than relying on syntax alone."

What Happens When an Agent Tool Call Fails?

A failed tool call should return to the model as a tool result marked as an error, with a message that names what went wrong and what to try next, so the model can fix its arguments or stop. Problems with the request itself, such as an unknown tool name, are protocol errors, which the MCP specification says models are less likely to be able to fix.

The MCP schema's comment on isError gives the reason: errors that originate in the tool belong inside the result object, "Otherwise, the LLM would not be able to see that an error occurred and self-correct." The Anthropic API has the same switch, an optional is_error field on the tool_result that you set to true when the tool execution resulted in an error.

• Say what to try next - Anthropic's handle tool calls page advises against a bare "failed" and gives "Rate limit exceeded. Retry after 60 seconds." as a better message.
• Name the field - a message such as amount_cents: expected integer tells the model which argument to change, which a generic validation failure does not.
• Expect retries - for an invalid or missing parameter, the same page says Claude will retry two or three times with corrections before apologizing to the user, so a tool can receive several attempts at one task.
• Set timeouts - MCP clients SHOULD implement timeouts for tool calls, but a timeout leaves a write in doubt: in the scenario the Idempotency-Key draft below opens with, a client whose POST timed out "is left uncertain about the status of the resource."

How Do You Make a Write Tool Safe to Retry?

Make the write idempotent: the caller attaches a unique key to each logical write, the tool stores the first result under that key, and a repeat with the same key and payload returns that result instead of writing again.

RFC 9110, the HTTP semantics standard, calls a method idempotent "if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request." It adds that a client "SHOULD NOT automatically retry a request with a non-idempotent method" unless it knows the request semantics are idempotent or can detect that the original request was never applied.

For POST-style writes, the IETF httpapi working group's Idempotency-Key header draft (version 08, dated 22 October 2026) defines a request header that makes non-idempotent methods such as POST or PATCH fault-tolerant. It is an Internet-Draft, which the IETF treats as work in progress rather than a standard, and its rules carry over to write tools:

• One key per write - the key "MUST be unique and MUST NOT be reused with another request with a different request payload," and the draft recommends a UUID or a similar random identifier.
• A repeat after completion - the resource should respond in a way consistent with the request being completed, which for a tool means returning the first result.
• A repeat while the first is running - the resource should respond with a resource conflict error.
• The same key with a new payload - the resource should reply with HTTP 422.

The key stops a duplicate only when every attempt at one write carries the same key. Your harness reuses it when it retries a timed-out call itself, but a call the model sends again is a new request, so generate the key in your code once per logical write, for example when the refund is approved, instead of letting the model make up a new one each time. If the model must pass the key, make the field required. The script takes the key as an argument, and its fake executor deduplicates like this:

// Excerpt from execute() in tool-call-check.mjs
const { idempotency_key: key, ...payload } = call.args;
const fingerprint = JSON.stringify(payload);
if (key && seenKeys.has(key)) {
  const first = seenKeys.get(key);
  if (first.fingerprint !== fingerprint) {
    return {
      isError: true,
      errors: [
        'idempotency_key reused with another payload',
      ],
      wrote: 0,
    };
  }
  return {
    ...first.result, deduplicated: true, wrote: 0,
  };
}

MCP's idempotentHint declares that "calling the tool repeatedly with the same arguments will have no additional effect on its environment," but it defaults to false and is a hint like every annotation. Only a test that sends the same write twice shows whether a tool keeps that promise.

Which Permissions Should an Agent Tool Get?

Give each tool only the functions and access its task needs, enforce authorization in the system the tool calls, and require a person to approve high-impact actions such as deletes, transfers and publishing. OWASP names the risk these controls address Excessive Agency.

OWASP's LLM06:2025 entry defines Excessive Agency as "the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction." It traces the root cause to "excessive functionality; excessive permissions; excessive autonomy," and its mitigations translate into rules for the refund agent:

The OWASP agentic list adds ASI02, Tool Misuse and Exploitation, for an agent that "operates within its authorized privileges but applies a legitimate tool in an unsafe or unintended way." One of its example attack scenarios is a refund that should never have been possible: "A customer service bot intended to fetch order history also issues refunds because the tool had full financial API access." Beside per-tool least-privilege profiles and human confirmation for high-impact actions, its mitigations add:

• An intent gate - "Treat LLM or planner outputs as untrusted," with a check of intent and arguments before execution.
• Immutable logs - "Maintain immutable logs of all tool invocations and parameter changes."

MCP security covers the specification's approval and confirmation rules for server tools and what changes once a model decides which of them run.

How Do Agent Tools Fail Inside an Agent?

Inside an agent, tools fail in ways a unit test of the tool cannot see: the model picks the wrong tool, fills the right tool with wrong or invented arguments, skips a call and reports the work as done, repeats a write, or follows instructions that arrived inside a tool result. Anthropic's engineering post Writing effective tools for agents says agents "might call the wrong tools, call the right tools with the wrong parameters, call too few tools, or process tool responses incorrectly."

Injected instructions need handling of their own, because tool results carry outside content:

• Untrusted results - Anthropic's handle-tool-calls page warns that tool results "often carry content from sources outside your control: webpages, inbound email, user uploads, third-party APIs," and tells you to treat that content as untrusted.
• Where the content goes - the same page says to keep untrusted content inside tool_result blocks rather than system prompts or plain user text blocks.
• Result validation - the MCP specification says clients SHOULD validate tool results before passing them to the model.
• Poisoned tool data - OWASP ASI02 lists external data tool poisoning, where "Malicious third-party content steers unsafe tool actions," among its common examples.

The claimed-but-not-done row has a guide of its own on agent action hallucination, which covers detecting false completion reports. The duplicate-write row, a refund retried after a timeout, also appears in MCP failure modes, which injects that timeout through a fault proxy and checks what the agent tells the user.

How Do You Design Tools an Agent Uses Well?

Build a few tools around the workflows the agent must finish, give each one a distinct purpose and unambiguous names, describe it the way you would brief a new hire, and return only what the model needs for its next step.

• Workflows over endpoints - Anthropic's engineering post says "More tools don't always lead to better outcomes," and for an address book it suggests a search_contacts or message_contact tool instead of list_contacts.
• One purpose per tool - the post asks for "a clear, distinct purpose" for each tool, while the define-tools page suggests grouping related operations, such as create_pr, review_pr and merge_pr, into one tool with an action parameter. Keeping reads and writes in separate tools keeps each tool in one effect class.
• Namespaced names - prefixes such as github_list_prs and slack_send_message separate services, and the post found that prefix versus suffix namespacing had "non-trivial effects" on its tool-use evaluations. MCP asks clients that combine servers to disambiguate colliding names, such as with a server prefix.
• Detailed descriptions - the define-tools page calls the description "by far the most important factor in tool performance" and suggests at least three or four sentences per tool, more for a complex one.
• Unambiguous parameters - "instead of a parameter named user, try a parameter named user_id."
• High-signal results - return the context the model needs and "eschew low-level technical identifiers (for example: uuid, 256px_image_url, mime_type)."
• Bounded responses - paginate, filter or truncate large results by default; the post notes, "For Claude Code, we restrict tool responses to 25,000 tokens by default."
• A small starting set - OpenAI's guide suggests fewer than 20 functions available at the start of a turn, "though this is just a soft suggestion," and an intern test: could a person use the function given only what the model sees?

How Do You Test Agent Tools?

Test agent tools in layers: unit-test each tool's code, contract-test its schema with the arguments models send, replay retries against every write tool, and run the agent on scenarios that compare the system of record with what the agent reported. Then repeat each scenario and gate releases on the results, since the τ-bench paper (June 2024) found function calling agents such as gpt-4o, then state of the art, "quite inconsistent" across repeated trials.

tool-call-check | 2026-11-05 | Node.js v25.5.0
Inputs: scripted, illustrative agent run, 8 calls
Tools: get_order (read), issue_refund (write)

[1] Schema: arguments checked before the tool runs
c1 get_order     PASS
c2 issue_refund  PASS
c3 issue_refund  PASS
c4 issue_refund  PASS
c5 issue_refund  PASS
c6 issue_refund  FAIL amount_cents: expected integer
c7 issue_refund  FAIL reason: "goodwill" not in enum
                      note: unknown field
c8 issue_refund  FAIL order_id: required field missing

[2] Retry safety: one write sent twice
c3 retries c2, key rf-1001-a  PASS deduplicated
c5 retries c4, no key         FAIL wrote twice

[3] Effect: what the agent said vs the ledger
ORD-1001 said 1 refund, ledger 1   PASS
ORD-1002 said 1 refund, ledger 2   FAIL duplicate
ORD-1003 said 1 refund, ledger 0   FAIL claimed, not done
ORD-1004 said 0 refunds, ledger 0  PASS
c1 get_order changed nothing       PASS

Summary: 15 checks, 9 PASS, 6 FAIL

How Can You Test Tool Calls and Their Effects With Agent Assurance?

TestMu AI's Agent Assurance covers the scenario layer for agents that act: agents that call tools, write files, update database records, hit APIs and open pull requests. It runs the agent you own against generated scenarios and checks each acceptance criterion against what the run left behind, such as records, files and observed tool calls, not against the agent's account of its work. Rook CLI, the rook command, runs it from your terminal or a CI job.

• Scenarios derived from your code - Rook CLI reads the agent's code, or its PRD or spec, and writes the scenarios; the one input you supply is how to invoke the agent: an HTTP endpoint, a command or an MCP tool.
• Tool calls checked against declared tools - the tool calls the agent actually made, as the profile's hooks report them, are checked against the tools the agent declares, including "must not call" rules; when no hook reports them, the scenario says so instead of guessing.
• Effects confirmed read-only - when the agent says it wrote a record, a judge can confirm the record exists with a read-only query through a tool you approve. Judges are told to verify without changing anything, because, as the Rook CLI README warns, calling issue_refund to find out whether a refund exists creates one.
• Adversarial scenarios by default - rook generate defaults to the functional and adversarial classes, and the adversarial categories include prompt injection, data exfiltration, hijacking and technical injection, mapped to the OWASP Top 10 for LLM Applications (2025), the OWASP Top 10 for Agentic Applications (2026) and MITRE ATLAS.
• Three verdicts - each criterion is Pass, Fail or Unable to Verify; a claim with nothing behind it stays unverifiable, and Unable to Verify is excluded from the pass rate's denominator, so it never counts for or against the agent.
• Headless runs in CI - export LT_USERNAME and LT_ACCESS_KEY from CI secrets, pass --yes once you have reviewed what the agent under test can do, and gate on the verdicts in rook report --json, because a finished run exits 0 whether its scenarios passed or failed.

Most eval and observability tools score what your agent said and recorded. Agent Assurance checks what the run changed, and reports what it could not verify. The Rook CLI README compares the default approach of each category:

The eval and observability columns describe each category's default approach, not any single product; several eval tools synthesize test cases, score tool calls, and ship red-team modules.

The agent's writes during a run are real, and Rook CLI cannot roll anything back, so point it at staging.

Agent Assurance is generally available, and its 14-day free trial includes 4,000 credits and the Pro features; Agent Assurance pricing covers the plans that follow it. Install Rook CLI from npm, which needs Node.js 20 or newer:

npm install -g @testmuai/rook@latest

Homebrew and the shell installer cover macOS and Linux on x64 and arm64 and need no Node, and npm also covers Windows x64. Tool-call checks depend on what a profile's hooks report.

What Should You Fix First in Your Agent's Tools?

Start with the write tool among your agent tools whose duplicate would cost most, such as a refund, a payment or an outbound email, and give it an idempotency key that your code generates once per logical write (or make the field required if the model must pass it). Then return an error result with a field-level message for every validation failure, and add a test that sends the same call twice plus one that compares the ledger with the agent's report. Those two checks caught ORD-1002 and ORD-1003 in the script.

After that, record each tool's effect class in your own code, put high-impact tools behind approval, and repeat your scenarios often enough to read pass^k instead of a single pass. For Rook CLI to check your agent's tool calls, set up the hooks that report them with the profiles and hooks guide.