Power Your Software Testing with AI Agents and Cloud
The Native AI-Agentic Cloud Platform to Supercharge Quality Engineering. Test Intelligently and Ship Faster.
- TestMu AI (Formerly LambdaTest)
- /
- Blog
- /
- What Is Jev? TypeSafe AI's System One Model Explained
What Is Jev? TypeSafe AI's System One Model Explained
Jev is TypeSafe AI's System One model: it returns typed decisions with calibrated probabilities instead of text. What it does, how to call it, where it fails.
Published on:
TypeSafe AI publishes an end-to-end response time of 70ms to 500ms for Jev, against 3 to 329 seconds for the frontier language models it benchmarks against, in its launch announcement.
Speed is one of three things the announcement leads with, alongside structured output and type safety, and all three come from the same design decision: Jev buys them by refusing to write anything. It has no string output at all, so there is no token-by-token generation to wait through and nothing to parse on the way back. You define the possible answers first, and the model fills them in.
Everything below is drawn from TypeSafe's own announcement and documentation, including the failure modes the company publishes about its own model.
Overview
Jev is the first System One model from TypeSafe AI. Instead of generating text, it evaluates typed questions against a block of state and returns a structured answer with a calibrated probability attached. Your code reads the value directly, with no parsing step and no risk of a malformed response.
Is Jev a Large Language Model?
No. Jev reads natural-language input the way an LLM does, but it generates no text at all: no replies, no code, no explanation of its reasoning. Giving up string generation is the trade that buys parallel sampling and an output type guaranteed before the call is made.
What Are Choice, Score, and Noul?
- Choice: Jev picks one option from a set you define in advance and returns the selected option, a probability for every option, and a confidence value. Options per question: up to 255.
- Score: Jev rates the state against ordered levels you write out, such as calm, frustrated, and very angry, returning a score, a probability per level, and a confidence value.
- Noul: Jev answers a yes/no statement with a single probability between 0 and 1. Separate confidence value: none, because the probability already carries that information.
- Parallel evaluation: Jev evaluates every question in a request independently against the same state in one pass, so adding more questions barely moves the response time.
- Input types: Jev accepts strings, JSON objects, and arrays of text. Image, audio, and video input: not supported.
Can You Act on a Jev Answer Automatically?
You can act on a Jev answer automatically only when it clears a confidence threshold you define. Jev returns confidence on every Choice and Score answer, so high-confidence answers proceed untouched, mid-band answers get a confirmation step, and low-confidence answers route to a person. TestMu AI gates agent quality scores the same way, attaching a confidence level to every metric it reports.
What Is Jev?
Jev is a model that takes a block of unstructured state plus a set of typed questions and returns typed answers with probabilities. TypeSafe describes it as a frontier-intelligence function call, and the shape of the API matches that description literally: state in, typed probabilistic decisions out.
The company announced it after two years in stealth. Founder Diogo Almeida previously worked at OpenAI on the instruction-following methods behind ChatGPT, and the announcement frames Jev as an answer to a question he describes carrying for four years: models have been superhuman at chat for years, so where is all the automation.
The technical claim behind that framing is a new training method, Reinforcement Learning for Calibrated Decisions, alongside a new architecture and a parallel sampler. Where reinforcement learning from human feedback optimizes for responses human raters prefer, this one optimizes for probabilities that match outcomes.
Four details shape what you can plan around today:
- Current release - the shipping version is jev-1.13.0. The alias jev-latest points at it and is the default in both client SDKs.
- Availability - early access behind a waitlist, with developers admitted in batches.
- The name - Jev is named after the economist William Stanley Jevons, whose paradox holds that efficiency gains increase total consumption rather than reducing it.
- Customization - there is no fine-tuning and no per-account weights. You shape answers through the state you send and the criteria you write, not through training.
What Is a System One Model?
A System One model is a class of model built to make fast, structured decisions that software consumes directly. TypeSafe's System One documentation defines it as a model that evaluates a state and returns typed answers and probabilities, and names Jev as the first System One model.
The name traces to Daniel Kahneman's Thinking, Fast and Slow. System 1 thinking is fast and intuitive, System 2 is slow and deliberate, and the model class is named for the first. TypeSafe acknowledges in its own announcement that System 1 thinking also carries a reputation for being error-prone, and argues the reliability question is separable from the speed.
Calibration is the property that distinguishes the class. A calibrated model's stated probabilities line up with how often it turns out to be right, so a batch of answers at a probability of four fifths should be correct roughly four times in five. The same TypeSafe documentation is careful about the limit of that guarantee: calibration is measured across groups of predictions, and it does not promise that any individual answer is correct.
A calibrated probability is something your code can threshold on and reason about statistically. An LLM's self-reported confidence, prompted after the fact, is a number it generated because you asked for one.
How Is Jev Different From an LLM?
An LLM produces strings for a human to read, and getting structured data out of one means constraining it, parsing the result, and validating that the parse succeeded. Jev removes those three steps by making the output type part of the request.
| Dimension | Frontier LLM | Jev |
|---|---|---|
| Output | Strings. Chat responses, code, refusals, or structured values that still need parsing and validating. | Type-safe structured values. Possible outputs are defined in advance and every answer carries a probability. |
| Sampling | Sequential. One token at a time, each conditioned on the last. | Parallel. All outputs generated in a single query. |
| Optimized for | Human preference, or programmatically verifiable rewards. | Calibrated decisions, meaning honest probabilities on System One tasks. |
| End-to-end latency | 3 to 329 seconds across the models TypeSafe measured. | 70ms to 500ms, which TypeSafe states as 40x to 200x faster on System One shaped queries. |
| Confidence | Self-reported when prompted, and prone to overconfidence. | Returned with every Choice and Score answer, derived from the probability distribution. |
| Input emphasis | Unstructured text, weighted toward sequential messages. | Unstructured text, weighted toward structured program state. |
The headline numbers deserve reading alongside the caveats TypeSafe attaches to them in the launch announcement linked above:
- 193.6x faster and 444.6x cheaper - these come from TypeSafe's own workflow evaluations, and the announcement states directly that they sit at the higher end of real-world gains.
- Reference answers are model-generated - the evaluations score against the average of two frontier models, which TypeSafe notes biases the comparison toward those vendors.
- The 0% type-error rate is not a measurement - TypeSafe writes plainly that the number is not empirical. Schema matching is guaranteed by the architecture, so 0% is a property of the design.
None of that makes the model slow or unreliable. It means the guarantee on offer is that Jev cannot return a value outside your schema, which is a narrower promise than never being wrong. Jev can still pick the wrong option out of the set you gave it, and the probabilities are how it tells you so.
What Questions Can You Ask Jev?
There are three question types: Choice, Score, and Noul. TypeSafe's primitives documentation calls them AI primitives, and you can mix all three in a single request against the same state.
Each question is evaluated in isolation. Adding a tenth question does not degrade the first nine the way stuffing ten instructions into one prompt degrades an LLM's output, and it barely changes the response time.
Choice
Choice selects one option from a set you define. The answer carries the selected option, a probability for every option in the set, and a confidence value. A single Choice question supports a cardinality of up to 255 options, and TypeSafe's own Wikiracing demo works around that ceiling with a two-stage approach that scores candidates first and then makes an explicit pick.
The full distribution is often more useful than the winner. Ranking retrieved passages by their probabilities, rather than acting on the top choice alone, is the pattern behind TypeSafe's re-ranking and passage-screening cookbooks, and it maps onto the retrieval problems covered in agentic RAG.
Score
Score rates the state against ordered levels that you write out as descriptions rather than as bare numbers. A three-level rubric of calm, frustrated but civil, and very angry returns a score such as 1.035, a probability for each level, and a confidence value.
A returned score is not a precise measurement, and the docs are explicit about it. TypeSafe states that score levels are weak in numerical calibration, so the expectation is safe to threshold against but not safe to interpolate between levels to reconstruct an exact figure. The 1.035 figure above is the worked example from TypeSafe's own quick start.
Noul
Noul asks whether a statement is true and returns a single probability between 0 and 1. There is no separate confidence field, because the probability carries that information already: 0.5 is the model telling you it has nothing to go on.
A Noul and a yes/no Choice on the same text do not return comparable numbers, and a Noul plus its own negation do not sum to 1. TypeSafe publishes a worked example where the two sum to 1.19, and tells you not to carry a threshold tuned on one question type over to another.
How Do You Call Jev?
Every call is a POST to a single endpoint, with a state field, a model field, and a questions object. Python and JavaScript SDKs wrap it, and the Python client reads the API key from the environment and defaults to jev-latest.
One request can ask all three question types about the same incoming bug report:
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
client = TypeSafeClient()
report = """Checkout button does nothing on Safari 17 after the latest deploy.
Console shows an uncaught TypeError. Repro'd on three machines. Our
Black Friday sale starts Friday."""
response = client.system_one(
state=report,
questions={
"component": Choice(
instructions="Which component owns this defect",
criteria={
"frontend": "Browser rendering, JavaScript, or CSS",
"payments": "Checkout, billing, or the payment gateway",
"infra": "Deploys, hosting, or networking",
},
),
"severity": Score(
instructions="How severe the reported defect is",
criteria=[
"Cosmetic, no functional impact",
"Degraded but a workaround exists",
"Blocking, no workaround",
],
),
"has_repro": Noul(
instructions="The report contains reproduction steps or a repro count",
),
},
)
# Each answer is read by key, typed to the question that produced it:
response.answers["component"].choice # one of "frontend" | "payments" | "infra"
response.answers["component"].probabilities # a probability for each of the three
response.answers["component"].confidence # 0.0 to 1.0
response.answers["severity"].score # a float across the three levels
response.answers["has_repro"].noul # 0.0 to 1.0The response shapes above are the ones documented for each question type rather than output from a run of mine, because access is still gated. The comments describe what each field holds, not values Jev returned.
The three questions cost one round trip between them, which inverts the usual instinct to bundle several judgments into one carefully worded prompt. TypeSafe's guidance is to decompose a broad judgment into atomic questions and combine the results with your own logic, so that changing a priority means editing a coefficient in code rather than rewriting a prompt.
The published model card for jev-1.13 puts the limits at 64k tokens per request covering the state plus all questions, with 32k covering the state plus the single longest question. The same card lists rate limits of 250,000 tokens per second and 1,200 requests per minute, with a warning that these are moving without notice while the company works through demand.
Access is the gate right now rather than the API surface. I called the endpoint without a key and got a clean, fast rejection:
$ curl -X POST https://api.typesafe.ai/v1/systemone \
-H "Content-Type: application/json" \
-d '{"state":"test","model":"jev-latest",
"questions":{"q":{"type":"noul","instructions":"Is this a test?"}}}'
{"detail":{"error_type":"authentication_error",
"message":"Must supply an API key! Check your request and try again."}}
HTTP 403 in 1.03sThe browser playground is gated the same way. Opening the console URL from TypeSafe's quick start lands on a sign-in screen rather than an editor, so the waitlist is the only route in until a key arrives.
What Does Confidence Tell You?
Confidence collapses the shape of the probability distribution into one number between 0 and 1, so you can threshold on it without doing the statistics yourself. A distribution concentrated on one outcome scores high, a flat one scores low. TypeSafe's confidence documentation notes that it is returned on every Choice and Score answer.
The recommended pattern splits confidence into three bands, each with different system behavior:
- High - act automatically, with no human in the path.
- Medium - proceed with a confirmation step, a review flag, or another question that gathers more information first.
- Low - do not act. Route to a person, ask for clarification, or fall back to a reasoning model.
Where you draw those lines is a risk decision, not a model decision. TypeSafe's own example gates a read-only balance lookup and a money transfer at different thresholds inside the same request, because showing the wrong screen is recoverable and moving money is not.
Scoring an AI output and then deciding whether the score is trustworthy enough to act on is the same loop that AI evaluation platforms run. TestMu AI's Agent Testing evaluates chat, voice, phone, and image agents across nine quality dimensions including hallucination detection, bias detection, completeness, and context awareness, with 30 or more additional metrics on phone calls. Each metric score carries its own confidence level, so a low-confidence evaluation surfaces for review rather than passing as a verdict. It is the same three-band gate the Jev docs describe, pointed at agent quality instead of at routing.
If the broader problem of scoring non-deterministic output is new to you, the fundamentals are covered in AI agent testing.
Where Does Jev Fall Short?
TypeSafe publishes a jaggedness page for jev-1.13 listing nine failure modes with a documented workaround for each.
Read it before you build rather than after. Most of the nine are avoidable by keeping the work in code that belongs in code, and the fixes cost nothing if you design for them up front.
| Failure mode | What goes wrong | Documented fix |
|---|---|---|
| Literal reading | Answers the question you wrote, not the one you meant. Negations and implied conditions are read at face value. | State the exact condition in the instructions and put boundary cases in the criteria. |
| Math and counting | Does not count reliably, and error grows with the size of the thing counted. | Ask one question per item and add the answers up in code. |
| Dates and times | Reads dates as text rather than ordered quantities, so ordering and windows are unreliable. | Extract date parts as Choice questions over closed sets, then assemble and compare in code. |
| Indirection | Accuracy drops across double negatives and multi-hop reasoning. | Write instructions directly and name the relevant parts of state. |
| Bloated state | Unrelated detail in the state acts as a distractor and costs accuracy. | Retrieve and filter in code first, or use a Noul to screen for relevance. |
| Adversarial content | State is treated as data, not as hostile, so injected instructions can move the answer. | Be explicit in the criteria and test edge cases before rollout. |
| Conflicting criteria | Instructions and criteria that ask for different things degrade the answer. | Treat criteria as an extension of the instruction and align the wording. |
| Structural invariants | Arithmetic identities you would expect to hold between questions do not hold. | Word each question to mean exactly what you want and do not port thresholds between types. |
| Generation | Not trained to produce text. Forcing it by chaining Choices is slow and works poorly. | Extract candidates with a regex or a generative model, then have Jev pick the right one. |
Two further constraints sit outside that table. English is the primary training language and where accuracy is best, with other languages handled less well, so non-English workloads need testing on your own content first. And the adversarial-content entry is a live concern rather than a theoretical one for anything that reads user-submitted text, which puts the same prompt-injection surface covered in LLM testing squarely in scope.
Note: Scoring AI output is only half the job. The other half is proving the score holds up across thousands of scenarios before you ship. TestMu AI runs that evaluation for chat, voice, phone, and image agents. Try it free
When Should You Use Jev Instead of an LLM?
The deciding question is whether you know the answer space before the call. If you can write the options, levels, or yes/no statements in advance, the decision fits Jev. If you cannot, you need a model that generates.
Jev is the better fit when your situation looks like this:
- The decision sits inside code - a branch, a route, or a filter where a multi-second model call is a bottleneck rather than a conversation.
- You need the uncertainty, not just the answer - gating on a calibrated probability is the difference between automating a task and automating most of a task and hoping.
- The same judgment runs at volume - classifying, screening, or scoring across a large corpus, where per-call cost and latency compound.
- The judgment decomposes - several independent factors you would rather weight yourself in code than bury inside one prompt.
Stay with an LLM when the output is text a person will read, when the task needs open-ended reasoning across many steps, when it involves arithmetic or date comparison, or when the set of valid answers is not knowable in advance. TypeSafe's own guidance points the same way, and the two are designed to compose: Jev decides, an LLM writes, and your code holds the thresholds between them.
Because Jev files no report of its own, verifying an agent it drives means grading effects rather than accounts, which is the subject of why Jev returns a type, not a sentence. Browser automation is the case where that split gets concrete, because Jev cannot fetch a page at all. Our measurements of how Jev acts on the web from a cloud browser cover how much of a real page survives extraction, and how often a page offers more links than one Choice question can hold.
Every performance figure published so far is vendor-reported, and no independent reproduction at scale had appeared as of this writing. That is normal for a model days into early access, and it is a reason to run your own evaluation on your own data before committing a production path, along the lines described in AI agent evaluation.
Getting Started With Jev
Join the waitlist at typesafe.ai, then pick one decision your code currently makes with a brittle rule or a slow model call and write it as a single Noul. One question, one state field, one probability back is enough to tell you whether the shape fits your problem before you design anything around it.
Once a decision layer is making calls on your behalf, the next question is how you know it is still right after the next model version ships. That is an evaluation problem rather than an integration one, and it is what TestMu AI's KaneAI testing agent and the quality metrics documented in the agent features and metrics guide are built to answer.
Author
Samyak Goyal is a Senior Member of Technical Staff at TestMu AI engineering Kane CLI, the command-line tool that runs browser automation from the terminal, where a flow described in natural language executes in a real Chrome browser and returns pass or fail with shareable proof. He is a backend engineer with 4+ years of experience, previously an SDE at Innovaccer, where he built APIs, introduced Kafka, and cut deployment from weeks to hours. Samyak also builds multi-agent systems, skill-orchestration frameworks, and a personal copilot that indexes 200+ microservice repositories.
Reviewer
Sirajuddin Khan is Vice President of Product Management at TestMu AI (formerly LambdaTest), where he drives the company's agentic AI product strategy, building a suite of autonomous agents that includes Agentic Browsers and Agentic Visual Testing and shifting the unit of work from test execution to autonomous outcomes. One of the company's earliest product leaders, he has owned the roadmap for the high-performance execution cloud and grew the cross-browser testing products from early adoption to market leadership. He brings over a decade of experience across SaaS, B2B, and eCommerce, with earlier product roles at Wydr and ShopClues, where his catalog and search work cut delivery SLAs and lifted seller activity. Sirajuddin holds an MBA in Information Technology from Sikkim Manipal University and a B.Tech in Computer Science Engineering from Maharshi Dayanand University.
Jev FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests





