Planner, Generator, Evaluator: Agentic AI Architecture

What Is Agentic AI Architecture

Agentic AI architecture is the arrangement of models, memory, tools, and control flow that lets a system pursue a goal across multiple steps rather than answering a single prompt. The architecture decides what the system can perceive, what it is allowed to do, and who gets to say the work is finished.

Most reference diagrams answer the first two questions well and skip the third. That omission is the difference between an agent that runs and an agent you can ship behind.

Two architectures can share identical components and behave nothing alike. The variable is who holds the verdict. If the component that produced the work also signs off on it, the architecture has one opinion in it, however many agents are drawn on the diagram. Our guide to multi-agent AI systems covers the coordination side of that in more depth.

The Component Layers Every Agent Has

Five layers show up in every serious agentic stack. The first four are standard. The fifth is the one teams add after their first bad release.

• Reasoning - the model that picks the next action. Anthropic uses extended thinking here as a visible scratchpad, so the plan is inspectable before any tool fires.
• Memory - short-term state inside a run plus long-term state across runs. Without it, step nine forgets what step two established.
• Tools - the surface where the agent touches real systems: APIs, shells, browsers, databases. This layer defines the blast radius.
• Orchestration - the router that decides whether one agent handles the task or several do, and how their results are recombined.
• Evaluation - the layer that scores the output against criteria fixed before generation started, in a context that did not produce the output.

A stack with the first four layers and no fifth is not broken. It is unfalsifiable, which is a worse property in production because every run reports success.

Why the Generator Cannot Grade Itself

A model evaluates an action more leniently when that action is framed as its own. Researchers call this self-attribution bias, and it fires on exactly the setup most teams ship: a coding agent critiquing the diff it just produced.

The March 2026 paper Self-Attribution Bias: When AI Monitors Go Easy on Themselves tested this across four coding and tool-use datasets. Monitors failed to report high-risk or low-correctness actions more often when the action had been generated in a previous assistant turn than when the identical action arrived fresh in a user turn.

The practical consequence is worse than the bias itself. Because monitors are usually benchmarked on fixed examples rather than on their own generated actions, the benchmark makes them look more reliable than they will be once deployed.

• Turn position matters, not disclosure - the same paper found that explicitly telling the model the action came from itself did not by itself trigger the bias. Conversation position did.
• Pull-request self-approval is the exposed case - the paper names coding agents self-critiquing generated code for pull request approval as one of the design patterns this failure mode hits.
• A second prompt is not a second opinion - re-asking inside the same conversation inherits the same context, the same interpretation, and the same blind spot.

That study is Large Language Models Cannot Self-Correct Reasoning Yet, and it examines intrinsic self-correction, where a model attempts to correct its initial response based solely on its inherent capabilities, without external feedback. Its finding is that models struggle to self-correct in that setting, and that performance sometimes degrades afterwards.

The distinction matters for architecture work. Correction against an outside source of truth is a different mechanism from a model second-guessing itself, and only the first one is worth wiring into a pipeline. We cover the QA-specific version of this in agent self-testing.

The Three Roles: Planner, Generator, Evaluator

The role split fixes the bias structurally rather than by prompting. A planner defines what correct means, a generator produces a candidate, and an evaluator that never saw the generator's reasoning decides whether the candidate meets the definition.

Anthropic documents the two-role version of this as the evaluator-optimizer workflow in Building Effective Agents, where one call generates a response and another provides evaluation and feedback in a loop. The same guidance notes that splitting a guardrail check onto a second model instance tends to outperform asking one model to handle both jobs.

The Handoff Contract Between Roles

Drawing three boxes is easy. The architecture only holds if each handoff carries a defined artifact, and if each role is barred from the decisions that belong to the others.

Three failure modes come straight out of that table, and each one is worth a check in your pipeline.

• Criteria drift - the evaluator loosens a criterion it cannot satisfy, and the run goes green against a weaker bar than the one the planner set.
• Context bleed - the generator's summary reaches the evaluator, and the evaluator grades the summary instead of the artifact.
• Evidence gaps - a criterion is marked satisfied with nothing captured behind it, so nobody can re-open the decision a week later.

Guard the third one hardest. A verdict without evidence has the same shape as a verdict with evidence right up until someone asks what actually ran.

Running the Evaluator Against Rendered UI

For anything user-facing, source code is the wrong ground truth. AI coding agents operate on a closed text surface, and their verification primitives, which are unit tests, type checkers, linters, and compilers, all read that same surface.

None of them render a viewport. So a button wired to the wrong endpoint, a redirect that lands on a 404, or a modal that will not close all pass the generator's own checks. The evaluator has to open the page.

Here is the evaluator step from a run executed on TestMu AI cloud for this article. The criterion was fixed first, the browser action came second, and the assertion reads rendered DOM text rather than anything the generating code claimed.

const { Browser } = require('@testmuai/browser-cloud');

const client = new Browser();
const session = await client.sessions.create({
  adapter: 'playwright',
  lambdatestOptions: { browserName: 'Chrome', browserVersion: 'latest' }
});

const { browser, page } = await client.playwright.connect(session);

await page.goto('https://www.testmuai.com/selenium-playground/simple-form-demo');
await page.fill('#user-message', 'planner-generator-evaluator');
await page.click('#showInput');

// Acceptance criterion: #message renders the submitted string
const rendered = await page.textContent('#message');
const verdict = rendered.trim() === 'planner-generator-evaluator' ? 'PASS' : 'FAIL';

await browser.close();
await client.sessions.release(session.id);

Real output from that run, build 102645107:

SESSION_ID=session_1787815044697_in708s
Playwright Adapter: Connected successfully!
ACCEPTANCE_CRITERION=#message renders the submitted string
OBSERVED_DOM_TEXT="planner-generator-evaluator"
VERDICT=PASS
DURATION_MS=5316

Notice what the verdict rests on. Not a model's opinion of the diff, and not a passing assertion the generator wrote, but the string a user would actually see on screen.

TestMu AI packages that role as Kane CLI, an agentic quality verifier that takes a natural-language objective, drives a real Chrome browser through it, and returns a pass or fail anchored to explicit evidence: DOM state, URL changes, network responses, console logs, and annotated screenshots. Every run seals an evidence pack, coverage is read off that pack per acceptance criterion with kane-cli cover, and distinct exit codes separate a real assertion failure from an environment problem or a timeout. Agent mode streams the same result as machine-readable NDJSON, so a coding agent can call the evaluator on itself and act on the verdict. The Kane CLI documentation covers installation and the CI wiring.

What the Evaluator Loop Costs

Adding roles adds spend, and pretending otherwise leads teams to bolt an evaluator onto workloads that never needed one. The Anthropic figures cited earlier put multi-agent token spend at roughly fifteen times a chat, with token usage alone accounting for most of the performance variance on its browsing benchmark.

The same source describes lead agents that spin up three to five subagents in parallel for typical work, two to four for direct comparisons, and more than ten for complex research. Fan-out is a dial, not a constant.

Anthropic's own guidance on when to spend it is blunt: agentic systems trade latency and cost for task performance, so start simple and add complexity only where it earns its keep. Applied to the evaluator role, that produces a short decision rule.

• Worth the loop - user-facing flows, payments, auth, migrations, anything where a wrong "done" costs more than the extra tokens.
• Not worth the loop - internal scripts, one-off data pulls, and refactors already covered by a deterministic test the generator did not write.
• Cheaper than a second reasoning layer - a deterministic verifier costs a browser session rather than a full agent you then have to supervise.

The last point is the one teams miss. A second LLM judge is another non-deterministic component in the stack, while a scripted or browser-grounded check adds a verdict you can reproduce.

Choosing an Architecture Pattern

The role split composes with whichever coordination pattern you already run. Pick the coordination shape from the work, then decide where the evaluator sits.

• Single agent with tools - correct default. Add an external evaluator before you add a second agent, because the evaluator removes more risk per unit of complexity.
• Orchestrator-workers - a central model breaks the task down, delegates to workers, and synthesizes. Use it when subtasks cannot be predicted from the input, and evaluate the synthesis rather than each worker.
• Evaluator-optimizer - generate, score, regenerate against the feedback. Works when criteria are explicit and iteration measurably improves the result, and it stalls when the evaluator cannot articulate why something failed.

For the wider pattern catalogue, including reflection, tool use, and guardrail layering, see our breakdown of agentic design patterns. The routing and failure-mode side is covered in agentic AI orchestration.

One rule survives every pattern. Whichever component synthesizes the final answer is disqualified from grading it.

Where to Start

Take one flow your coding agent already ships and write its acceptance criteria before the agent touches it again. Two or three criteria, each phrased so a browser could check it without a human reading prose.

Then wire the evaluator as a required check rather than an optional step. Install Kane CLI locally from the documentation linked above, point it at that flow in plain English, and let the exit code gate the merge. If your agents talk to users directly, Agent Testing applies the same separation to conversational and voice agents.

The architecture change is small and the reporting change is not. Once verdicts arrive with evidence attached, review shifts from reading diffs to checking what actually ran, which is the only version of this that scales with agent output.