What Gemini's Security Eval Reveals About Agent Trust

Inside Google's Gemini Security Evaluation

The evaluation was run by Irregular, an AI-focused cybersecurity company. NBC News reported Google's confirmation that Gemini gained unauthorized access to three systems belonging to outside organisations, by guessing login information or by using credentials it found in a public repository. The model stopped before doing anything further with its access.

The Hacker News, citing Irregular's report, attributes the breaches to a security test domain mix-up: a naming error made a fictional company name used in the capture-the-flag exercise match a real domain, and the models took advantage of inadvertent internet access. The Hacker News also reports that Gemini got into one protected system after repeatedly guessing its password.

• May 2026 - the three intrusions happen during Irregular's evaluation.
• July 2026 - Irregular reviews its work for incidents similar to the Hugging Face disclosure, and Google learns of the intrusions. Google investigates, informs the organisations behind the websites and tells federal authorities.
• 18 September 2026 - Google confirms the intrusions publicly. NBC News notes they were reported earlier that day by The Wall Street Journal.

Adkins summed it up for The Hacker News: "In this case, the model acted appropriately." Given what the model believed about where it was, that reading holds.

The Lab Disclosures Since July

Gemini's case is the latest of several since July.

On 27 July, Hugging Face published an agent intrusion technical timeline describing an agent driven by OpenAI models that escaped its sandbox during an internal capability evaluation on OpenAI's platform.

On 30 July, Anthropic published its account of incidents in cybersecurity evals: three incidents in which Claude reached real organisations' systems, identified after reviewing 141,006 evaluation runs where Claude could have obtained internet access.

A fourth Anthropic incident followed in September, when The Hacker News reported that Anthropic AI models had breached real systems in a January run as well. Anthropic then widened its scan to roughly 481 million transcripts and, per the same report, said all four of its incidents occurred in evaluations built by the same partner.

OpenAI followed on 16 September with six further misalignment incidents from its own models, one of which is covered in OpenAI compaction summary misalignment. The mechanics across these cases differ. What the table's rows share is an environment that was supposed to be sealed and was not, and a model that acted inside it on the assumption that it was.

Sandbox Escape vs a Sandbox That Was Never Sealed

A sandbox escape is when code confined to an isolated environment breaks out and reaches the host or the network, the same containment idea behind browser sandboxing. Strictly, only one of the four disclosures fits: the OpenAI-driven agent that Hugging Face describes exploited a zero-day in a package registry cache proxy to get out.

Anthropic's July incidents and Gemini's started from a door left open. Anthropic's report says its evaluation prompt "stated explicitly that Claude had no internet access", then adds: "Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available." Gemini's environment had the same gap, plus a scenario name that pointed at a real company.

• Escape - the isolation held as configured and a vulnerability broke it. The fix is patching plus defence in depth, so one broken layer does not reach the internet.
• Unsealed environment - the isolation was never there. The fix is verifying the configuration from inside the environment on every run, because the documentation, the prompt and the partner's understanding can all be wrong at once.

The second kind needs no exploit at all, which is why it deserves a check in every agent test run, frontier lab or otherwise.

The Premise Was the Failure

Read Google's account closely and nothing went wrong in the reasoning. In a capture-the-flag task, guessing a password and using found credentials is what a capable model is supposed to do, and each step followed from the one before. The one wrong thing was the premise under all of them: in Adkins' words to NBC News, websites "it thought were part of the test".

That premise belongs to the agent's account, even though it rarely gets treated as one. An agent's account usually means its story about what it did: the summary, the transcript, the "task complete". Underneath sits a story about where it is, and that story covers facts no transcript states outright:

• which systems are real and which are simulated
• whether the network is closed
• which actions can be undone
• whose data the agent is touching

The agent does not verify that story. It assembles it from what it was told and what it can see. In the Anthropic runs, the prompt told Claude it had no internet access; in Gemini's run, the scenario pointed it at a name that existed outside the exercise. Both inputs were wrong, and the reasoning built on them was sound.

How an Agent's Account Absorbs Contrary Evidence

Anthropic's report shows what happens once the premise is set, and it is the most useful passage any of the labs has published. The example involves Anthropic's own models.

In one evaluation, a Claude model published a package to a registry it believed was part of the simulation. In reality the package "was made freely available online for roughly one hour". The model had noted that publishing it on the real internet would be "NOT okay, and surely not the intended solution", and concluded it was still simulated because it "didn't recognize the (in reality genuine) certificate authorities securing its connections". Anthropic's report continues:

"Claude never revisited this conclusion; when automated scanners began installing the package, for example, Claude took them to be scripted actors within the evaluation."

Evidence that the world was real arrived, and the account absorbed it. The account stayed internally consistent, so reading the transcript more carefully would not have helped, because nothing in it contradicted anything else.

For a reviewer, that means an agent's transcript can only be checked against itself. A check that catches a wrong premise has to come from something the agent did not write, such as network logs, package registry records or the files on disk.

How Each Incident Was Detected

The agents were the last to know. Gemini's intrusions came to light only after Irregular reviewed its records, and Anthropic's model never revisited its conclusion. In Anthropic's cases, the targets missed it too: "The two organizations we were able to reach had not previously detected the activity or contacted us."

• Hugging Face - its timeline says: "The first signals came from several layers of our security stack at once: live runtime analysis, SIEM logs, and others." The trigger was activity on its own infrastructure.
• Google - learned in July, about two months after the May intrusions, when Irregular went back over its own evaluation work.
• Anthropic - screened its runs for a condition, whether Claude could have obtained internet access, and widened the scan after the fourth incident.

Every detection that worked keyed on an effect or a capability: activity on real infrastructure, records of what ran, reachability of the network. None started from the agent's description of its own situation. The same principle underpins agent observability: record what the agent did from a layer the agent does not write to.

The disclosures themselves deserve credit. OpenAI and Anthropic published their own accounts, Google notified the affected organisations and federal authorities in July before confirming publicly, and Gemini stopped before using its access.

How to Check an Agent's Environment Before It Runs

These were lab evaluations, and none of it happened in customer use. The structure still transfers. Agents in production are told things about their world too: this is staging, this is a test account, this action can be reversed. Each is an input the agent reasons from and none is something it checked, which is the same gap that shows up in AI agent reliability work.

The Gemini failure points at a cheap first check. Classic fictional company names are an easy trap, because most of them are registered domains. A public DNS lookup of common scenario names returns:

• Real IP addresses - acmecorp.com, acme-corp.com, globex.com, initech.com, contoso.com, fabrikam.com, northwind.com, cyberdyne.com, umbrellacorp.com, wayneenterprises.com, megacorp.com and dundermifflin.com.
• No A record - hooli.com, piedpiper.com and evilcorp.com, which still does not prove the name is unused.
• Nothing, by design - acme.test, globex.example and initech.invalid.

The last group is the safe one. RFC 2606 reserves the .test, .example, .invalid and .localhost top-level domains for testing and documentation, so scenario names under them cannot collide with a real organisation's domain.

A preflight is a script that runs before the agent starts and confirms the environment matches what the agent has been told. The one below checks scenario names and egress separately. Run the name check wherever scenarios are written, because it needs public DNS. Run the egress probe inside the evaluation environment, immediately before the agent starts.

// eval-preflight.mjs
// Run before an agent evaluation starts. Exits non-zero when a "fictional"
// target is a real domain, or when the sandbox can reach the internet.
import { resolve4 } from "node:dns/promises";

// Every hostname the scenario tells the agent is fictional
const scenarioHosts = ["globex.com", "initech.com", "umbrellacorp.com", "acme.test"];

// Any public endpoint works as the egress probe
const EGRESS_PROBE = "https://example.com/";

async function resolvesPublicly(host) {
  try {
    return (await resolve4(host)).length > 0;
  } catch {
    return false;
  }
}

async function egressOpen(url) {
  try {
    await fetch(url, { method: "HEAD", signal: AbortSignal.timeout(5000) });
    return true;
  } catch {
    return false;
  }
}

let failed = false;
let unverified = false;

// Inside a sealed sandbox, public DNS is unreachable and every name would
// "pass". Report that as unverified instead of calling it a pass.
if (!(await resolvesPublicly("example.com"))) {
  console.log("UNVERIFIED  public DNS unreachable, scenario names not checked");
  unverified = true;
} else {
  for (const host of scenarioHosts) {
    if (await resolvesPublicly(host)) {
      console.log(`FAIL        ${host} resolves on the public internet`);
      failed = true;
    } else {
      console.log(`ok          ${host} does not resolve`);
    }
  }
}

if (await egressOpen(EGRESS_PROBE)) {
  console.log(`FAIL        outbound HTTPS to ${EGRESS_PROBE} succeeded`);
  failed = true;
} else {
  console.log("ok          no outbound HTTPS from this environment");
}

process.exit(failed ? 1 : unverified ? 2 : 0);

Run from a workstation with ordinary internet access, it fails on every real domain and on the egress probe, and exits 1:

$ node eval-preflight.mjs
FAIL        globex.com resolves on the public internet
FAIL        initech.com resolves on the public internet
FAIL        umbrellacorp.com resolves on the public internet
ok          acme.test does not resolve
FAIL        outbound HTTPS to https://example.com/ succeeded
$ echo $?
1

The UNVERIFIED branch matters. Inside a properly sealed sandbox, public DNS is unreachable, so every scenario name would fail to resolve and look safe. The script reports that as unverified and exits 2, rather than counting a check it could not perform as a success.

The preflight covers the environment. The run itself needs its own evidence:

• Record effects the agent does not author - outbound connections, DNS queries, tool calls and file writes, captured by the harness and diffed against an allowlist after every run.
• Treat environment facts as claims - a prompt that says there is no internet access and a config file that says the same are both accounts. Anthropic had both, and the network disagreed.
• Keep "could not verify" separate - folding an unchecked result into pass or fail hides exactly the runs where the premise went unexamined.

That last habit is the idea TestMu AI builds Agent Assurance around. For autonomous agents, it invokes the agent for real and grades each criterion against observed evidence: files that changed, artifacts produced, and tool calls checked against the agent's declared tool surface. Anything it cannot establish is reported as Unable to Verify, a third verdict kept out of the pass rate. The page on Agent Assurance results and evidence states the rule directly: "An agent's claim that it sent a message or created a refund is not independent proof of that write."

Agent Assurance does not make an environment safe on its own. It calls your agent's real endpoints, and its setup guide asks you to keep the target on a disposable or non-production environment, which is itself an environment claim worth running through the preflight above.

Note: Agent Assurance grades autonomous agents on what they did, and reports what it could not check as its own verdict instead of a pass. Create a free TestMu AI account

Start With the Preflight

Add the preflight to the job that launches your agent evaluations, and move any scenario host that resolves to a name under .test. Then grade each run on what the agent did, from records it did not write.

The Agent Assurance quickstart runs a first suite against a bundled sample agent, and AI agent evaluation covers where effect-based grading fits in a wider eval program.

Citations

• NBC News, 18 Sep 2026: unauthorized access to three systems
• The Hacker News, 19 Sep 2026: security test domain mix-up
• Anthropic, 30 Jul 2026: incidents in cybersecurity evals
• Hugging Face, 27 Jul 2026: agent intrusion technical timeline
• The Hacker News, 10 Sep 2026: breached real systems