Dawei Li

Dawei Li

Graduate student in NLP & Information Retrieval | Research: Emotion Analysis, Hierarchical Classification | University of Tsukuba

Jev，一个不说话的 AI：关于“System One 模型”的笔记Jev, the AI that does not talk: notes on a “System One” model話さない AI、Jev：「System One モデル」についてのメモ

约 5 分钟4 min read約 6 分

Published:

On September 15 a startup called TypeSafe AI came out of stealth with Jev, which it calls the first System One model. Its CEO, Diogo Almeida, is a former OpenAI researcher and a co-author of the InstructGPT paper; the company has a $40M seed round. Jev is in early access through a hosted API.

The pitch is unusual enough that I spent the week trying to decide whether it is a new category or a well-packaged classifier. My answer is “both”, and the interesting part is not the one on the landing page.

What it does

Jev never writes text. You send it a piece of context (a “state”) and one or more typed questions, and it returns values of those types:

  • choice — one option from a list, with a probability for every option and a confidence;
  • score — a value on a defined scale, with probabilities;
  • yes/no — a probability between 0 and 1.

Several questions about the same state are answered in a single parallel pass. TypeSafe quotes 70–500 ms end to end, $0.042 per million input tokens, output free, and “40–200× faster than frontier LLMs” on this kind of task. It was trained with what they call RLCD — Reinforcement Learning for Calibrated Decisions: the goal is that when Jev says 95%, it is right 95% of the time. There is no image, audio or video input yet.

event / ticket /agent step Jev (System 1)choice · score · yes/no+ probability, 70–500 msseveral questions, one pass deterministic codehigh confidence, low risk LLM (System 2)open-ended, needs reasoning humanlow confidence or high stakes p ≥ τ(risk)
How I would place it: a fast decision layer that sends each case to code, an LLM or a person.

Reading the claims carefully

“Zero hallucinations” is true by construction and a little misleading. Jev cannot produce an option that does not exist, so it cannot break your schema. It can still pick the wrong option. Typed output removes format errors, not judgement errors.

The benchmarks are against LLMs. According to heise, the reference labels in TypeSafe’s evaluation are the average of GPT-6 Astra’s and Claude Fable 5.1’s answers; Wavect’s review notes the vendor’s own four-workflow test and its “444.6× cheaper” figure. So Jev is, at best, a model that agrees with frontier LLMs at a fraction of the latency and cost. That is valuable — but it is distillation economics, not a claim to be more correct than them.

The real product is calibration. A router that is right 90% of the time is useful. A router that knows which 10% it is unsure about is far more useful, because you can send exactly those cases elsewhere. That is the promise of RLCD, and it is also the claim you have to verify yourself, because calibration only holds on the distribution it was measured on.

stated confidence observed accuracy 01 calibrated: “95%” means 95% over-confident illustration, not measured data
What "calibrated" means. Before trusting a threshold, draw this chart on your own traffic.

Where it fits: the small decisions inside agents

In my harness comparison I argued that the harness is now where agents differ. Look inside any harness and you find dozens of tiny decisions per task: which tool, whether to retry, whether a step failed, whether to ask the human, whether an output is safe to show. Today most of those are either hard-coded rules or another full LLM call that takes seconds and costs as much as the real work.

That is the gap Jev targets: System 1 for the reflexes, System 2 for the thinking, code for the rules. The sensible pattern — also what Wavect recommends — is to keep arithmetic, permissions and side effects in code, run Jev in shadow mode against your current logic first, and set the confidence threshold per risk: a wrong FAQ route and a wrong refund decision do not deserve the same bar.

A connection to my own research

My graduate research was on hierarchical emotion labels, where predicting amusement for a joy example is a smaller mistake than predicting sadness (EEMD). Routing has the same structure: sending a billing question to the refunds queue is a near miss; sending it to security is not.

A model that only returns its top choice cannot use that. A model that returns calibrated probabilities over all options can: instead of taking the most likely option, you choose the one with the lowest expected cost, using a cost matrix built from the label hierarchy.

OptionJev probabilityCost if the true class is billingCost if refundsCost if securityExpected cost if chosen
billing0.460140.96
refunds0.401041.02
security0.144403.44

(Illustrative numbers.) Here “billing” and “refunds” are close, so the expected cost of choosing either is low, while a confident-looking but wrong jump to “security” is expensive. This only works if the probabilities mean what they say — which is exactly the property TypeSafe is selling.

My reservations

  • Hosted only, for now. In private-AI work, the first question is where the data goes. A small, fast decision model is the ideal thing to run on-premises (it is the kind of workload that fits a NIM-style container well); until that exists, regulated customers will not put their tickets through it.
  • Weak on numbers. Dates, prices and exact calculations are not what it is for.
  • Adversarial input. The state is user text. A decision model that routes support tickets is also a target for people who want to be routed somewhere.
  • Confidence is not permission. 99% sure is still not authorised.

Jev feels like the first product built on an idea I have had for a while: that most “AI decisions” in production should not be generated at all, only chosen — quickly, from a known set, with an honest probability attached.

Sources

You May Also Enjoy