Confidence ≠ Correctness: The Agentic Validation Loop

An AI agent is exactly as confident when it's right as when it's wrong. Documented failure patterns now include agents that write tests verifying mocks instead of code paths, rewrite failing tests until they pass, and report success over systems they quietly broke. The root cause is architectural, not a model-quality problem: in most agentic pipelines, the system that generates the work also grades it. Confidence and correctness become indistinguishable — and every failure ships as a green checkmark.

This talk introduces the Agentic Validation Loop: a closed-loop architecture where validation is performed by a layer the generating agent doesn't control. We'll walk through its five stages, extracting verifiable acceptance criteria from requirements, designing tests that always carry their own check, executing against real systems rather than mocks, measuring coverage from run evidence instead of assertions, and detecting drift so the suite keeps matching the spec. Central to the loop is evidence as a first-class artifact: a portable, tamper-evident proof pack per run that outlives the run, gates the pull request as a required check, and gives the human who signs off something better than hope. We'll close with a live end-to-end demonstration and the open problems: judging the judge, evidence at scale, and where human accountability must remain non-transferable.