Jev Returns a Type, Not a Sentence - and That Changes How You Check Your Agents

What Is Jev, Exactly?

The framing TypeSafe uses is Kahneman's. System Two is slow deliberate reasoning, the thing we've spent three years buying by the token. System One is fast intuitive judgment, the kind that happens before you notice it happening.

Their argument is that most of what production systems ask a model to do is System One work wearing a System Two price tag. Route this ticket. Score this lead. Is this within policy. Does this need a human. Each is one judgment with a small answer space, and each has been going through a chat-shaped model that returns a paragraph you then parse.

Jev does the judgment and skips the paragraph. TypeSafe lists the obvious homes for it: support routing, lead scoring, trust and safety checks, document classification, candidate filtering, workflow branching. They also demoed compacting a million tokens of context to about 86,000 in roughly a second, context management as a typed selection pass rather than a generative summarisation pass.

If you build agents, you already know where this lands. Not as the agent, but as the fast layer inside it. The router in front of the expensive model. The policy gate before the tool call. The is-this-safe check that was too slow to sit in the hot path and now isn't.

That's the part to think carefully about.

How Have We Actually Been Checking Agents?

Be honest about the current method. Teams read the transcript.

They skim what the agent said it was doing, look at the reasoning trace, and form a judgment about whether it sounds right. Evals get built on top of that: LLM judges reading agent output, humans spot-checking conversations, dashboards counting how often the text looked acceptable.

It's a weak method, and most people half-know it. An agent's account of what it did is the weakest evidence available about what it did, because it's generated by the same process that took the action, with no independent access to whether the action was correct. But weak-and-legible still gives you a surface. You can read prose. You can catch it out.

Here's one from an expense-approval agent we test against at TestMu AI. Asked about claim EXP-1002, it replied:

approved for $450. This is within the $200 limit for software.

Read it twice. The sentence contains the two numbers that disprove it. The agent narrated its own policy violation in the act of committing it, and that narration is the only reason a human reviewer catches this in thirty seconds.

That's prose-era assurance. The lie is inspectable because the lie is written down.

What Happens With a Typed Model in That Seat?

Same agent, same claim, but the policy check is now a fast typed call, which is exactly where it belongs architecturally. The output is this:

{ "approved": true, "confidence": 0.94 }

Where is the tell? There is no sentence holding the claim amount next to the policy limit for a reviewer to notice. The decision is wrong in precisely the same way and it leaves no readable trace. Nothing to skim, nothing for a judge model to read, nothing that looks off in a transcript review.

Typed output does not make agents fail more often. It makes them fail silently.

Silent failure is the expensive kind. A loud failure gets caught in review. A quiet one ships, runs at 200 milliseconds a call for a quarter, and surfaces as a metric nobody can explain: approval rates drifting, a support queue mysteriously mis-routed, a policy that turns out to have been advisory all along.

The Confidence Score Is the Same Self-Report, Wearing a Lab Coat

This is where teams are going to fool themselves, and it's worth being blunt about it.

Jev returns calibrated confidence. 0.94. It looks like measurement. It has a decimal point. It invites you to set a threshold and feel principled, fast path above 0.9 and escalate below it, and that is a sensible engineering pattern.

But look at what the number is. It's the model's assessment of its own reliability, produced by the model, in the same forward pass as the decision. That's a self-report. It's the agent's account of what it did, compressed from a paragraph into one float, and the compression made it look more like evidence, not less.

Calibration is real work and a calibrated score genuinely beats a vibe. Useful for routing and sufficient for assurance are just different claims, and they're about to get conflated at scale, because a float is so much easier to put on a dashboard than a paragraph.

None of this is a knock on Jev. Jev is the clearest argument anyone has made this year for verifying agents by their effects, made accidentally, by a company that wasn't arguing about testing at all.

What Is Left When There Is Nothing to Read?

One thing: what the agent actually did. The tool calls it made, the rows it wrote, the refund it issued, the claim it approved. Effects do not narrate and do not self-assess. They either happened or they did not, and this is the part that matters, they are checkable by something other than the agent.

Concretely, for anyone building this quarter:

• Make the agent observable before you make it fast - an agent that appends every tool call to an audit log can be verified without trusting a word of its output. One whose only record is its own transcript cannot, and when that transcript becomes a single true, it cannot be verified at all.
• Treat every typed call as a test surface - it returned a valid enum value is a type check. It returned the right one is an assurance question, and it needs cases with known-correct answers and a check against effects.
• Never let a confidence score stand in for a verdict - thresholding on self-reported confidence is a routing decision. A dashboard of average confidence is a dashboard of the model's self-esteem.
• Say unable to verify when you cannot - the honest third verdict matters more as outputs get terser. A system that reports everything as pass or fail, when half the effects were never observable, is generating false assurance, which is worse than none because someone will act on it.

Note: Grading an agent on its effects needs the effects captured in the first place. TestMu AI Agent Assurance invokes the agent for real and reads what changed, rather than asking it what happened. Create a free account

The Short Version

An agent's account of what it did has always been the weakest evidence about what it did. Jev removes even that account, and the industry is going to install it in thousands of decision points this quarter because it is fast and nearly free, which are good reasons.

Which leaves exactly one thing worth checking. Not what your agent says. What it did.

Citations

Jev's latency, pricing and calibration claims are TypeSafe's own, reported by the following outlets rather than measured here.

• TechCrunch: A new kind of AI model from a ChatGPT inventor is thrilling developers
• AI News: ChatGPT pioneer launches Jev model for programmatic logic
• heise: AI model Jev to make machines decide faster