What Is an Agentic Quality Engineer? Role, Skills and Workflow

What Is an Agentic Quality Engineer?

An agentic quality engineer is a quality engineer who works through AI agents and on AI agents. The role directs agents that write, run and triage software tests, and tests the AI agents the team ships. It also builds the rules, permissions and CI gates that this work runs under, and owns the evidence behind each release decision.

• Agents as testers - a coding agent such as Claude Code and a testing agent such as KaneAI or Kane CLI write, run and repair tests of web, mobile and API layers. The engineer decides what they test and approves the result.
• Agents under test - the chat, voice, phone, video and image agents a product uses to talk to customers, and the agents that call tools and change records. Both are tested for what they say and what they do.
• The quality setup - CLAUDE.md conventions, permission rules, hooks, eval suites and CI gates decide what an agent may run on its own and what a human must approve.

The title is new, and employers define it in different ways. TOPdesk's Senior Agentic Quality Engineer opening, read on October 26, 2026, splits the job into the same three parts: "testing agentic product behaviour", using agents to test conventional software, and building the quality infrastructure behind both. It also names Claude Code as the team's main agentic engineering environment.

Several related terms describe parts of this work:

• Agentic testing is about how software gets tested, with agents planning, running and healing the tests of an application.
• Agent testing is about what gets tested, with the AI agents your team builds as the system under test.
• Agentic QA is the working practice in which the agent is handed a goal rather than a script and chooses its own steps.
• An AI QA engineer tests AI features and uses AI tools for the rest of the test work. The agentic quality engineer is that job with agents doing most of the execution.
• Quality engineering builds quality in from the requirements onward, so the acceptance criteria an agent tests against come before the code.

In Anthropic's study Measuring AI agent autonomy in practice, about 20% of new users' Claude Code sessions ran with full auto-approve, rising to over 40% as users gained experience. Experienced users also interrupted Claude more often, in about 9% of turns against 5% for new users. As fewer actions pass a human checkpoint before they run, the quality setup has to be designed in advance.

What Does an Agentic Quality Engineer Do?

The duties start from a habit most developers already have. In the Stack Overflow Developer Survey 2026, 76.5% of 13,162 respondents run an AI-generated answer locally before using it. Of the same respondents, 63.9% compare it against the codebase and 53.0% inspect tests or security implications, while only 9.6% use it as is.

An agentic quality engineer turns that habit into a team process. The duties employers list for agentic and AI quality roles fall into directing testing agents, testing the agents a team ships and maintaining the agent rules, permissions and CI gates.

How Is an Agentic Quality Engineer Different From a QA Engineer or an SDET?

The three roles share test design, risk thinking and defect reporting. They differ in who writes the tests and what counts as proof.

A QA engineer combines manual and automated testing, and an SDET writes the framework code that automated tests depend on. An agentic quality engineer has agents draft and run most of the tests, then spends the time saved on specifying, reviewing and preparing the release evidence.

The shift follows where code now comes from. In JetBrains research on how much code developers let agents write, over half of developers now write less than 20% of their code by hand, and one in five writes none without AI help.

What Skills Does an Agentic Quality Engineer Need?

Checking AI output is the human skill workers rank first. Microsoft's 2026 Work Trend Index asked 20,000 workers who use AI which human skills matter more as AI takes on more work. Quality control of AI output came first at 50%, followed by critical thinking at 46%.

The matrix draws on 11 job postings for agentic quality, AI quality and agentic engineering roles, read on October 26, 2026. Agent-specific skills come first, and each skill has one drill for a side project. Some of them, such as judge calibration and red teaming, appear in only one posting.

The QA base, from test design to defect reporting, still applies; the QA engineer guide covers it.

Which Tools Does an Agentic Quality Engineer Use?

The coding agent is usually the first layer. In the AI section of the Stack Overflow Developer Survey 2026, Claude Code (66%) and GitHub Copilot (59%) lead the coding agents developers used in the past year.

TestMu AI's testing agents come as a platform and as a terminal tool:

• Platform testing agent - KaneAI tests web, mobile and APIs. It turns a Jira ticket, PRD, PDF, screenshot or spreadsheet into test cases and runs them across browsers and real devices on HyperExecute. When the UI changes it heals steps, and each heal is open to review. Its tests export to Selenium, Playwright and Appium code.
• Terminal testing agent - Kane CLI runs where a coding agent can call it. With its skill installed, Claude Code, Codex CLI or Gemini CLI runs a plain-English objective with kane-cli run or replays designed _test.md files with kane-cli testrun run. The coding agent reads the NDJSON result and the evidence, then decides whether to keep going, fix the code or bring the failure to you.
• Design from requirements - kane-cli context ingest reads a PRD or story, and kane-cli design tests writes acceptance criteria, scenarios and one runnable _test.md file per scenario.
• Evidence per run - each run leaves an evidence pack with per-step screenshots, an annotated copy of each, browser console and network logs, run logs and failure records.
• Coverage against criteria - kane-cli cover reports which acceptance criteria the evidence has proven, so a green run with an unproven criterion is visible.

Note: Kane CLI is free to install. Its runs use KaneAI credits from your TestMu AI account, on your own machine or on the TestMu AI cloud grid, and you can start free with 10,000 credits, valid for 30 days, which you claim on the Kane CLI page. Create your TestMu AI account

For the agents a product puts in front of customers, Agent Testing tests chat, voice, phone, video and image agents:

• Scoring by agent type - simulated users hold the conversations and calls, and each agent type is scored on its own metrics, such as hallucination and bias for chat and voice agents and more than 30 metrics for phone calls.
• Command-line runs - agent-testing-cli runs chat and phone caller evaluations from a terminal or a CI job.

Agents that call tools, write files or change records need a different check: what the run changed. The section on testing AI agents that take actions covers it.

How Does an Agentic Quality Engineer Work in Claude Code?

Sessions led by more expert users succeed more often. Agentic coding and persistent returns to expertise, Anthropic's analysis of about 400,000 Claude Code sessions, counted a session as a verified success only with a hard signal such as a passing test suite. Novice-rated sessions reached it 15% of the time, against 28% to 33% for sessions rated intermediate or above.

The session that follows is an example:

• A team ships a web shop plus a refund agent that issues refunds through a tool call.
• Commands come from the Claude Code, Kane CLI and Rook CLI docs, and file contents and prompts are illustrative.
• No test was run for this guide, so no run output appears.

# Testing rules

- Unit tests: `npm test`. Keep them green.
- Design E2E tests with the kane-cli skill from
  requirements/*.md. Never hand-write them.
- Designed tests live in .testmuai/tests/.
- Test against staging only:
  https://staging.example.com
- Test the refund agent (agents/refund/) with
  /rook against staging.
- Ask before any Kane CLI run and before anything
  that writes to a real system.
- Never weaken an assertion to make a test pass.
- Report Pass, Fail and Unable to Verify apart.
- A human signs off every release.

npm install -g @testmuai/kane-cli
npx @testmuai/kane-cli-skill
npm install -g @testmuai/rook@latest
npx @testmuai/rook-skill@latest install \
  --agent claude-code

$ claude --version
2.1.283 (Claude Code)
$ kane-cli --version
0.9.0
$ rook --version
0.1.13

{
  "permissions": {
    "allow": [
      "Bash(npm test *)"
    ],
    "ask": [
      "Bash(kane-cli context ingest *)",
      "Bash(kane-cli design *)",
      "Bash(kane-cli run *)",
      "Bash(kane-cli testmd run *)",
      "Bash(kane-cli testrun run *)",
      "Bash(rook *)"
    ],
    "deny": [
      "Read(./.env)",
      "Read(./.env.*)",
      "Bash(git push *)"
    ]
  },
  "hooks": {
    "PostToolUse": [{
      "matcher": "Edit|Write",
      "hooks": [{
        "type": "command",
        "command":
          "${CLAUDE_PROJECT_DIR}/.claude/hooks/unit.sh",
        "args": []
      }]
    }]
  }
}

#!/bin/bash
# Runs after Claude edits a file. Exit 2 sends
# the failure text back to Claude.
FILE=$(jq -r '.tool_input.file_path // empty')
case "$FILE" in
  *.ts|*.tsx|*.js|*.jsx) ;;
  *) exit 0 ;;
esac
cd "$CLAUDE_PROJECT_DIR" || exit 0
if ! npm test --silent > /tmp/unit.log 2>&1; then
  tail -n 20 /tmp/unit.log >&2
  exit 2
fi
exit 0

/kane-cli Design E2E tests from requirements/refunds.md
for https://staging.example.com. Ingest the file, show
me the use-cases to approve, then design at most 8
tests. Stop before any run and list the variables I
need to fill.

kane-cli context ingest requirements/refunds.md \
  --mode agent
kane-cli context review --approve <uc-ref>
kane-cli design tests --use-case <uc-ref> \
  --mode agent --max 8

  --mode <mode>
      Ask policy: interactive | agent | ci | override. TTY
      defaults to interactive chat; REQUIRED when headless

kane-cli testrun run --match 't-' < /dev/null
kane-cli cover gaps

/rook Inspect this repository and my Rook CLI setup.
Confirm the loaded skill and its references, the CLI
version, authentication, the selected project and
agent, the profiles and the available credit balance.
Explain what is missing. Do not install or update
software, change definitions, invoke the target, or
run paid operations yet.

Use the rook skill to run the one smoke scenario I
selected with the staging profile. Confirm its ID,
any writes it can make to the target and its credit
use before asking for my approval. Keep the run ID
and the evidence. Do not retry or start a paid
root-cause report on your own.

name: e2e
on: [pull_request]
jobs:
  kane:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: '24'
      - uses: browser-actions/setup-chrome@v2
        id: chrome
      - run: npm install -g @testmuai/kane-cli
      - name: Replay designed tests
        env:
          KANE_CLI_CHROME_PATH: >-
            ${{ steps.chrome.outputs.chrome-path }}
          KANE_USER: ${{ secrets.KANE_USER }}
          KANE_KEY: ${{ secrets.KANE_KEY }}
        run: >
          kane-cli testrun run --headless
          --username "$KANE_USER"
          --access-key "$KANE_KEY" < /dev/null

unzip -q .testmuai/evidence/<execution_id>.evidence \
  -d review/kane
rook report --json > review/rook-report.json

---
name: evidence-reviewer
description: Reviews test evidence after test runs
tools: Read, Grep, Glob
---

You review test evidence and never edit files. For
each failed or unverifiable check, quote the saved
evidence and the requirement it traces to. Flag
tests with no assertion and runs with no run ID.
Never call a check passed that the tool did not
pass.

How Does an Agentic Quality Engineer Test AI Agents That Take Actions?

The README in the public LambdaTest/rook repository on GitHub puts it this way: "An agent's account of what it did is the weakest evidence available about what it did." Read the agent's tool calls and the records it changed from evidence it did not write. AI agent verification covers the identity, authority and action checks in depth.

Behavior under test can also differ from behavior in use. Anthropic's Agentic Misalignment study was a deliberately constructed simulation with a 2025 model. In it, Claude Opus 4 chose blackmail in 55.1% of the samples where it stated the scenario was real and in 6.5% where it stated it was an evaluation.

Those figures come from a lab scenario rather than production traffic, and they argue for tests that resemble the environment the agent will run in.

Agent Assurance from TestMu AI tests agents that act, such as coding, workflow and back-office agents, through its terminal client, Rook CLI. Rook CLI derives scenarios from the agent's code or spec, adversarial ones included, runs the real agent against staging or inside CI, and judges each acceptance criterion from evidence:

• Tool calls - the calls that the hooks in your Rook CLI profile (separate from Claude Code hooks) report for each scenario, compared with the tools the agent declares and with any tool a criterion rules out.
• Side effects - what those hooks observe outside the agent, such as files written under declared paths, artifacts, and read-only probes through a tool you approve.
• Claims - an action that appears only in the agent's own report never counts as evidence, although model judges do score some criteria.

Most eval and observability tools score what your agent said and recorded. Agent Assurance checks what the run changed, and reports what it could not verify. The same README compares tool categories by their default approach rather than by product, and notes that several eval tools also generate test cases, score tool calls and include red-team modules.

• Verdicts per criterion - every criterion gets one of three verdicts: Pass, Fail or Unable to Verify. Unable to Verify is neither a pass nor a failure: it marks a gap in what the profile can observe.
• Staging only - send every run to staging. Rook CLI invokes your real agent, so a refund issued during a test is a real refund, and in the README's own example a judge calling issue_refund to see whether a refund exists would create one.

As the Rook CLI README describes, exploring, generating, writing profiles and running all spend Agent Assurance credits, and since version 0.1.13 rook checks that the account has an Agent Assurance plan or an active free trial before any model call or credit spend. The 14-day free trial includes 4,000 credits with the Pro features, and the Agent Assurance plans are on the pricing page.

The docs note that commands and stored file formats can change between releases. Install Rook CLI and its Claude Code skill:

npm install -g @testmuai/rook@latest
npx @testmuai/rook-skill@latest install \
  --agent claude-code

• npm route - needs Node.js 20 or newer and is the route for native 64-bit Windows. Rook CLI runs on macOS and Linux, and 64-bit Windows through npm or WSL.
• Homebrew and the shell installer - cover macOS and Linux and need no Node.
• Claude Code skill - the installer needs Node.js 22 or newer. Start a new Claude Code session in the agent's repository, type / and pick rook.

The Rook CLI Claude Code setup guide covers the full setup and how to update the skill.

What Is the Career Path to Agentic Quality Engineer?

Employers post the title in several forms, and most ask for years of test or quality engineering first. The postings in the table were read on October 26, 2026, and are not linked, because job pages expire.

Read together, the postings suggest this path:

• Entry point - a QA engineer or SDET who owns automated suites and starts reviewing agent-written tests.
• Practitioner - an agentic quality engineer who writes agent rules, runs evals in CI and tests the agents the product ships.
• Lead or architect - sets release-readiness criteria and quality gates across teams, as the Bridge AI, Zayo and FAiHr postings describe.

How quality engineering teams may reorganize around agents is discussed in What a QE Org Will Look Like in 2027.

No official pay figure exists for the title yet. The closest official benchmark is the US Bureau of Labor Statistics profile for software quality assurance analysts and testers. It gives a median annual wage of $104,300 in May 2025 and projects employment to grow 6%, from 187,600 jobs in 2025 to 198,200 in 2035.

How Do You Become an Agentic Quality Engineer?

Learn with the agent instead of only handing work to it. In Anthropic's randomized trial How AI assistance impacts the formation of coding skills, 52 mostly junior engineers learned a new Python library. The group using AI scored 50% on a quiz right after the task, against 67% for the group coding by hand, a gap of 17 points.

The participants who showed stronger mastery used AI to ask follow-up and conceptual questions while they worked. Apply the same habit to each step below, starting from a QA engineer or SDET role:

• Practice on a side repository - give a coding agent a small app with tests, ask it to add a feature, and read every test it writes before you accept the change.
• Write the testing rules - put your test conventions into CLAUDE.md, including the rule that no assertion may be weakened to pass, and run /context to confirm the file loaded.
• Review agent-written tests on real work - take agent-written tests from your team's pull requests and trace each one to the requirement it claims to cover.
• Add one hook and a permission set - make the fast tests run after each edit, and put credit-spending runs, pushes and writes to real systems behind an ask or deny rule.
• Test one agent your team ships - run an evaluation of a chat or phone agent, or point Rook CLI at a staging copy of an agent that acts. Then write up what the run could and could not verify.
• Own one release decision - collect the runs, the evidence packs and the unverified checks for one release, then write down the go or no-go call and the reason for it.

For a longer learning plan across AI testing topics, the AI roadmap for software testers lays out the phases in order.

To start as an agentic quality engineer this week, install Kane CLI and its skill, open a side repository in Claude Code, and have the agent test one user flow in a real browser while you read every assertion it writes. For the agents you ship, the Agent Testing CLI docs show how to run chat and phone caller evaluations from the same terminal.