Decision engineering, Peter Milan Trapp

Judgement is becoming cheap.

Every company makes thousands of small judgements a day. Is this complaint urgent? Is this report complete? Does this invoice match the order? Most are made on a sample or too late, because a person has to make them. A new kind of AI model makes them in about half a second, for a fraction of a cent, and says how sure it is.

One million judgements
People2 minutes each
$1.5 millionabout 20 person-years
Chat model2 seconds each, writes a sentence
3,300× cheaper than people
$455measured with Claude Haiku
Decision modelhalf a second each
88,000× cheaper than people27× cheaper than the chat model, saves $1.5 million
$17measured with Jev
Change the two values to your own. One block is one person-year of 1,700 hours. Chat model and decision model measured on the same fault report, September 2026; Jev at its short-text rate.
01What is different

A new kind of model that decides, and says how sure it is.

A chat model writes an answer. A decision model, the first of which is Jev from TypeSafe, picks one of the answers you allow and says how sure it is. Below, both get the same fault report at the same moment.

1Input data

Fault reportLine ID attachedCustomer offline since 07:40Line test: no light at the fibre boxAccess next week, date to be agreed

2Question / prompt

Is this report complete enough to send a technician?

System One modelpicks one of Dispatch Send back Cannot tell
Chat modelanswers in one or two sentences
System One modelJev
3Output
0.00 s
Send back0.87
Dispatch0.10
Cannot tell0.03
4Threshold

0.87 is below the 0.90 you set

5Action
Software actsA person decides
$17per million cases
Chat modelClaude Haiku 4.5
3Output
0.00 s

No, this fault report is not complete enough to dispatch a technician. Key information is missing, including the specific Line ID (it says “attached” but isn’t shown), the exact nature of the fault (no light at ONT could indicate multiple issues), and crucially, no confirmed appointment date/time has been scheduled with the customer for next week.

4Threshold

No number to compare

5Action

Someone has to read the sentence

$455per million cases
4.6× faster27× cheaperwhen the chat model writes a sentence; about 5× for a one-word answerSame conclusion: send it back
Plays on repeat, 3 times slower; the bars under step 3 show real seconds. Median of five calls per model, 28 September 2026.

Where it is strong

  • About half a second and $17 per million decisions
  • A probability with every answer, so software can act on it
  • The same input gives the same answer

Where it is weak

  • It writes no text and gives no reason
  • Its probabilities need checking on about 50 of your own cases
  • It runs in the US only: personal data is pseudonymised first, or an open model runs on your own servers

TypeSafe’s own claims next to the measurements →

02Tested

On short texts, close to a large chat model at a fifth to a seventh of the cost. On long contracts, 4 points behind.

Four public test sets, each run through Jev, a large chat model (GPT-5.6 Luna) and a free model that runs on your own hardware, with the same instructions, September 2026.

TestRight answersCost per million decisionsTime per case
JevGPT-5.6 LunaFree modelJevGPT-5.6 LunaJevGPT-5.6 Luna
News sorting300 public news articles, each sorted into one of four sections89%87%86%$17$870.5 s2.0 s
Telecom requests300 customer requests to a telecom operator, routed to one of seven teams95%98%83%$20$1440.4 s2.4 s
One detail flips the answer100 pairs such as “please cancel” and “please don’t cancel”100%100%58 to 72%$17$870.5 s2.0 s
Full contracts300 checks of full confidentiality agreements against fixed statements80%84%48%$90$7200.4 s2.8 s

GPT-5.6 Luna: a large chat model from OpenAI. Free model: a similarity model that runs on your own hardware, so it has no cost per decision. Cost and time for the pairs follow the news test, since the texts are of the same length. The telecom requests are synthetic, written for chatbot training (Bitext, CDLA-Sharing licence). In these four tests the large chat model cost 5 to 8 times as much per decision as Jev; on longer support emails, about 11 times.

Watch the telecom test run300 requests per model, the same request in the same square in both grids
60× faster
Jev
Time0 s
Cost$0.0000
Right–

GPT-5.6 Luna
Time0 s
Cost$0.0000
Right–

RightWrongGoes to a person
Each grid answers the 300 requests one after another at its measured time per request, played 60 times faster. In production both run in parallel; the ratio stays the same.

Method, data and the independent tests others published: the details.

03Calibration

Not plug and play: each question needs a threshold calibrated on your own cases.

A threshold only works if “90% sure” means right about 90% of the time. That holds on familiar tasks and slips on unfamiliar ones, so every question goes through a short calibration before it runs on its own.

News test: how often Jev was right, by how sure it was
0.90 to 1.00 sure, 246 cases92.7% right
0.80 to 0.89 sure, 18 cases66.7% right

Calibration decides how much runs on its own. In the telecom test, 90% of the requests cleared 0.90 and 98.9% of those were right; the large chat model cleared 0.90 on 75%. In the news test, 82% of the articles needed no person. On tasks the model has not seen, independent testers found calibration 2 to 4.4 times worse. Where a company’s categories differ from the plain meaning of the words, a model can be very sure and still wrong; only a check on your own cases shows it.

  1. Check the labels firstDo two people give the same answer on 30 to 50 cases? Where they disagree, no model will be reliable, and the categories need cleaning before anything else.
  2. Label about 50 past cases per questionfrom your own data. For a rare outcome, about 300 cases with at least 100 of the rare one.
  3. Compare and correctthe model’s probabilities against the true answers. A one-line correction usually brings them back in line.
  4. Set the threshold from what an error costsA wrong automatic dispatch costs a wasted visit; sending back a complete report costs the customer a day.
  5. Pin the model version and check againbefore a new version takes over the same question.

Method and sources: how the tests are run.

04What is possible

What you could decide on every case.

Pick a department. Each card is one decision: the question, the answers you allow, and what one million of them cost. The cost is the measured price; how often each decision is right is tested on your own cases before it runs.

Costs as measured. Short texts such as messages and reports: about $17 per million with Jev, $87 with a large chat model. Full documents such as contracts: about $90 against about $720. At 2 million fault reports a year the model bill is about $34 with Jev, so the money lies in the staff hours on the cases that no longer need a person. Mark the decisions that fit your company; the first conversation starts with them.

Companies already report results.

Four companies in their own figures, within weeks of the release.

Software

Unblocked

Their assistant answers engineers' questions from internal notes. Jev decides which notes belong in each answer.

More of the right notes and fewer wrong ones, in production, for under two dollars per thousand questions.

Source
Insurance

FurtherAI

An AI agent fills in insurance quotes on broker portals. Jev picks each next step.

A quote that took 18 minutes now takes about 5, and the setup is moving into production.

Source
IT services

CinchOps

A small IT firm sorts customer reviews and search queries by topic.

5,300 reviews sorted for 14 cents instead of 50 dollars, as accurately as the expensive model.

Source
Retail pricing

Pricogni

A pricing tool matches competitors' products so prices can be compared.

9,081 product pairs checked for 32 cents. The unclear ones go to a person.

Source
05Better or differently

Where would cheap judgement pay off for you?

Pick the one closer to your situation; the next step starts there.

Where does your company make the most of these judgements?

06First questions

What usually comes up in the first meeting.

Data protection and works council

Jev runs on US servers. Personal data is pseudonymised before a case is sent, or an open model of the same kind runs on your own servers. The details list what a works council usually asks.

The people who decide today

Below the threshold every case still goes to a person. People set the thresholds, handle the unclear cases and check a monthly sample of the automatic ones.

Your systems

Each decision is one call from the system you already use: a ticket, a dispatch or an invoice step. The wiring for one decision is part of the Decision Test.

Model versions

The default version can change without notice. The threshold is set on a pinned version, and a new version goes through the same check before it takes over.

Cost beyond the model

The model bill is small. The work lies in labelling about 50 cases per question, the integration and the handling of unclear cases; the Decision Test prices this for one decision.

Where it is not proven yet

Four public tests are measured, one of them on telecom requests. In German, results were a few points lower than in English, so a rollout in German is tested in German. Each decision in your company is tested on your own past cases before anything runs on its own.

More on data protection, works council and IT: the details.

Option 1: A call

Talk one decision through with Peter.

30 minutes on one decision in your company. If it fits, three steps can follow, and you can stop after any of them.

  1. Decision Discovery90 minutes

    3 to 5 decisions worth testing

  2. Decision Test1 to 2 weeks

    Your past cases, a go or no-go

  3. Production Pilot4 to 8 weeks

    One live decision

Reply within 48 hours. No customer data needed. Data protection, works council and IT

Portrait of Peter Milan Trapp
Who wrote this
Peter Milan Trapp

Strategist and builder for AI and new ventures, with startup experience. Peter works inside your team, from the first question to a process that runs in production. Email: peter.milan.trapp@gmail.com, LinkedIn

  • Tested independently. Every comparison and calculation on this page comes from own runs through the public interface, without TypeSafe’s involvement. Results of other companies are their own figures.
  • Pilots and workshops in several industries on where cheap judgement changes decisions, processes and teams.
  • A workshop kit inside the lab, so a team can run the inventory on its own.