Executive Summary

Everything we hand a generative model looks, from the outside, like talking. Look at what actually happens inside a pipeline and a good deal of it is not talking at all. Is this email spam? Which team should this ticket go to? Should this agent action be escalated to a person? Each one picks a single item out of a small set fixed in advance, and we have been asking a sentence generator to do the picking. Jev, released on 15 September 2026, does the picking and nothing else. By the vendor's own description it cannot write prose, cannot write code, and cannot hold a conversation.

What it gives back instead comes in three shapes: Choice, a selection from the options you supply; Score, one level on a rubric you define; and Noul, a yes or no. All three arrive with a probability attached to the answer. The differentiator the company leads with is not speed or price but that probability. TypeSafe says the model was trained to get the probability of a judgment right rather than to match human preference, and the name it gives the technique is reinforcement learning for calibrated decisions, or RLCD. Producing an answer a person will read and producing an answer a program will branch on are different jobs — that is the design premise the company argues from.

1.1Three Answer Shapes, Each With a Number Attached

The three output primitives are built so that calling code can branch on them directly. The table below sets out what each one takes in and what it returns. The last column is the one section 4 spends the most time on: a probability comes back with every answer.

The three output primitives TypeSafe published. In all three the set of possible answers is fixed by the caller before the request. Input is priced at $0.042 per million tokens, and output has no billing line at all: the company's pricing page reads "FREE (too cheap to meter)."

Free output reads like marketing copy, but structurally it is closer to a statement that there is nothing left to bill for. A generative model builds its answer one token at a time, and each token has to wait for the one before it, so a longer answer costs more and takes longer. A model that only picks has no sequential generation step. The candidate answers already exist, so the work is distributing probability across them, which means latency tracks the length of the input rather than the length of the output. The numbers the company puts in its own write-up point at that structure: end-to-end response time of 70 to 500 milliseconds for Jev, against a range of 3 to 329 seconds for the frontier models it compares with. TypeSafe has not published the model's architecture or training method, so everything in this paragraph describes what a picking-only design implies in general, not something verified inside this particular product.

1.2Never Leaving the Option List

One benefit of the design is beyond dispute: the output format cannot break. Any team that has pushed classification through a generative model knows the failure modes — an apology where the label should be, a paragraph of explanation wrapped around the JSON you asked for, a category name that exists nowhere in your taxonomy. With a picking-only model none of that can happen, because the only things available to pick are inside the list.

On content the picture is not uniformly bad. One independent evaluation, built by someone with no vendor involvement, put Jev's zero-shot accuracy at 95.9% on a 400-item benchmark of its own construction. On the same task a hand-written keyword rule scored 77.2%, and a TF-IDF logistic regression trained on every label scored 66.0%. On a narrowly defined task, in other words, a model given not one line of training data beat a supervised classifier. The same evaluation also produced a result pointing the opposite way, which section 4 takes up.

1.3A New Category, or an Old Job in New Packaging

The sharpest reaction to the launch was not about performance but about category. Anastasios Angelopoulos, chief executive of the model evaluation platform Arena, gave the Financial Times the version most widely quoted.

"It's unclear to me what makes these models different from standard 'zero-shot classifiers', which are relatively well-known technology."

Anastasios Angelopoulos, chief executive of Arena, in the Financial Times, 25 September 2026. The original sits behind a paywall and this report verified the wording through a mirror. In Korea, AI Times carried an objection to the same effect on 26 September.

On the facts the objection holds up. Tools that attach labels without writing prose are already in wide use. The table below lists what a team could reach for today.

The competitive field for picking-only tools. Mistral's Shieldstral is the closest in spirit to Jev's Noul: instead of retraining a fixed harm taxonomy, it lets the rule itself be expressed as a question.

The counter-argument rests on the same set of facts. Nobody, it runs, had packaged three things together as one product before: several shapes of structured answer, many questions fired in parallel against the same state, and training aimed at calibration rather than at accuracy. On the Hacker News launch thread the most upvoted criticism was that calling this a frontier model borrows credibility the product has not earned; a reply in the same thread argued that the framing is marketing but accurate marketing, since Jev is the reflex and the deliberation is meant to stay in your code or in a real LLM.

The category question does not get settled here on the industry's behalf. What stands on the record is that the industry split on a shared set of facts, and the decision moves in section 5 to the reader's own pipeline. Because that decision cannot even be posed until the price and the accuracy are measured properly, sections 2 and 3 come first.

Collect the figures the documents covering this launch actually used and you get: 86x per call, 171x, a four-hundredth of the price, 444.6x, a range of 4.8x to 238x, and 580x. Every one of them describes the same model in the same week. They do not contradict each other. What changes each time is what got divided by what. The size of the multiple is set by the comparison model, not by Jev.

This report therefore writes the numerator and the denominator next to every price claim it makes. The table below does that for the multiples now in circulation. Read the comparison column and the reason the figures spread across more than a factor of twenty stops being a property of the model and becomes a property of the comparison.

Seven multiples describing one event. The largest is roughly 120 times the smallest. None of the arithmetic is wrong. The denominators differ.

2.1Dividing the Vendor's Own Table

The firmest ground here is the workflow evaluation dashboard TypeSafe published itself. Nine models were run across four business tasks, with cost per call, latency and accuracy laid out on one screen, and the per-task figures reconcile arithmetically with the aggregates. Jev's four task costs of $0.0001, $0.0003, $0.0011 and $0.0001 average to the published $0.0004, and Opus 5 reconciles too.

All nine models from the vendor's workflow evaluation, re-sorted by cost per call. Every value is the arithmetic mean of the four tasks. Orange marks the extreme in each column, grey marks the bottom. That the highest cost and the highest latency belong to two different models is where the diagram below starts.

Sorted by price, the multiples from the previous table look different. The model one row above Jev is not Opus 5 but GPT-5.6 Luna. Luna runs $0.0033 per call, a little over eight times Jev; it takes 12.9 seconds, some thirty times longer; and it scores 66.9%, 0.9 points below Jev. Inside the vendor's own table, that is to say, the model nearest to Jev on accuracy delivers roughly the same score for a single-digit multiple of the price, not a three-figure one. The 440x holds only if the comparison model is Opus 5, and choosing Opus 5 is done by whoever writes the sentence, not by the data.

The same table shows one more thing. Both models that serve as denominators for the headline multiples are themselves beaten inside it. Opus 5 ($0.1761, 37.8s, 73.1%) loses to Sol ($0.0836, 23.3s, 74.1%) from the same vendor on cost, latency and accuracy alike. Sonnet 5 ($0.1174, 78.1s, 67.8%) loses to Terra ($0.0304, 10.1s, 67.9%) on all three as well. The two cells no practitioner reading this table would actually pick are the two that generate the largest multiples.

2.2The Comparison Splits Inside a Single Line

The company's home page hangs two numbers side by side: 193.6x faster and 444.6x cheaper. Nowhere in the company's documents is it written what either number is measured against. The Batch carried the gap forward as it stood, describing Jev as 193.6 times faster than unnamed large language models. Run both figures back through the aggregate table and the cost multiple fits Opus 5, the most expensive model in the set, while the latency multiple fits Sonnet 5 at 78.1 seconds. Both residuals come in under one percent and point the same way, which is what rounding in the displayed values would produce.

The latency side needs one more qualifier. What matches 193.6x is Sonnet 5's 78.1 seconds, but the slowest of the nine models is not Sonnet 5; it is DeepSeek v4 Pro at 86.5 seconds, which divides out to 216x and misses the headline. So the denominator of the speed claim is not the slowest model in the table but the second slowest, and it is the slowest only if you restrict the field to the OpenAI and Anthropic families. The cost claim needs no such qualifier. Opus 5 really is the most expensive of the nine.

The point is not to corner the company. In the same post TypeSafe characterised these numbers itself.

"This is where the claims of 193.6x faster, 444.6x cheaper on our home page comes from, and we expect that these are on the higher end of real world gains."

TypeSafe's official blog, 15 September 2026. The company states in its own words that the headline multiples sit at the top of the range of real-world gains.

Go one step further and the multiples turn out to diverge within a single site, not just between documents. Open the company's home page on 27 September and below the fixed headline a pair of live examples cycles through. Those figures are $0.000081 against $0.013880 per call and 0.114 seconds against 8.566 seconds, which divide out to 171.4x on cost and 75.1x on latency. Both sit at roughly two-fifths of the 444.6x and 193.6x printed on the same screen, and the comparison in the live example is labelled only "LLMs," with no model named. The company's blog uses a third framing as well: a range of 40x to 200x faster for system-one style queries. The "up to 200x" that reached Korean coverage is the top of that range, not a rounding of 193.6x.

A number the company described as the high end of the range ends up on the front page, and by the time it reaches a news article the qualifier has fallen away. That is why every multiple in these pages arrives with a denominator. Inflation usually happens not by inventing a figure but by stripping the conditions off one that is real.

2.3The Same Value Differs by Document, and Unevenly

Put the per-call costs cited by The Batch beside the aggregates on the company dashboard and a third problem appears. The two documents disagree, and they disagree by different amounts on different lines. A unit difference or a blanket rounding would push all three values the same way by the same proportion.

Sonnet agrees for practical purposes; the other two are off by nearly a factor of two. Neither document contains a formula that would connect the three values.

For a practitioner the lesson is specific. Whether a figure comes from a vendor announcement or a news article, copying it into a procurement memo means copying the document and the version date along with it. Open the same table two months later and a changed number is then an update, not an incident.

The line that travelled furthest was about accuracy rather than price: 67% on the company's own dataset, on a par with the large models. That sentence leaves two things out. What was the 67% matched against, and who exactly is the model on a par with? The company's own published material answers both.

Start with the answer key. Nobody decided what counted as correct in this evaluation. The averaged verdict of two larger models was installed in the correct-answer slot, and the company wrote down both that fact and the bias it introduces.

"We use the average of GPT-6 Astra and Fable 5.1 as the reference answer, which biases answers towards OpenAI and Anthropic's models."

"However, they were made by individuals on our model capabilities team, so some bias could exist."

TypeSafe's official blog. The first sentence concedes how the answer key was built and that it tilts results toward two specific vendors; the second concedes that the evaluation tasks themselves were written in house. Together with the "higher end of real world gains" line quoted in section 2, all three sentences appear in the same post.

Accuracy in this evaluation therefore means agreement with the averaged verdict of two larger models, not agreement with answers a person supplied. The 67.8% measures how closely Jev picks the way those two models pick, rather than how often it picks correctly. On any item where both graders were wrong together and Jev was right, the table records Jev as wrong.

3.1The Two Models That Set the Answers Sit at the Top of the Price List

What kind of models wrote the answer key is something the price list will tell you. Line up September 2026 rates and the two graders are at the top, with the nine models being graded below them. The 238x input-price multiple the company advertises has exactly this teacher price in its numerator.

Rates as of September 2026. The benchmark was run on 15 September, so Opus is listed at its pre-cut price. One table holds the whole shape of it: the two most expensive models decide what is correct, and the model that matches them 67.8% of the time costs 238 times less on input.

3.2Who the Parity Claim Is Actually About

The second omission is the comparison set. On the four-task average Jev scores 67.8%, Sonnet 5 scores exactly the same 67.8%, and Terra scores 67.9%. As a statement about totals, parity is true. But both of those models are in the lower half of the frontier field in this evaluation. At the top of the same table sit Sol at 74.1% and Opus 5 at 73.1%.

Per-task accuracy from the vendor's evaluation table, in percent. Orange marks the leader in each row. The full table carries nine models including the DeepSeek v4 family and Luna; the six discussed in the text are shown here.

Read task by task and what the average was hiding comes out. The gap is entirely different from one task to the next, and on one of them it reverses. The chart below sets Jev against the leader on each task.

Overlay the widest gap, invoice processing, with the cost side of the same source data and something else surfaces. That task is the most expensive and the slowest for all nine models. Opus 5 spent $0.4856 per call on it, four to eight times what it spent on its other three tasks, and Sonnet 5 took 241.3 seconds to handle a single item. Jev is no exception: at $0.0011 per call, invoice processing was the costliest of its four tasks too. The task where the saving looks largest is the task where the accuracy is cut most. Look only at the averaged line, without breaking the saving out per task, and those two facts never appear in the same cell.

3.3What Shifts When a Model Sets the Answers

Handing the grading to a model is not something this product invented, and the literature has names for what the design disturbs. The canonical reference, the 2023 MT-Bench work, showed that a strong judge model reaches roughly the level of agreement humans reach with each other, and in the same breath reported three biases: position bias, where the order of the candidate answers moves the verdict; verbosity bias, a preference for longer answers; and self-enhancement bias, a tendency to reward answers from the judge's own family.

Self-preference became a research topic of its own after that. Work published from 2024 onward measures how much a judge rates its own output more highly, and notes that separating genuine preference from the possibility that the answer really was better is itself a methodological problem for the field. A comprehensive survey from late 2024 lengthens the list further with authority bias, majority-opinion conformity, and verdicts that shift depending on whether the source was anonymised.

The literature carries one implication here. If the answer key is the average of an OpenAI model and an Anthropic model, models that answer the way those two answer will score well. That is precisely the bias TypeSafe conceded, so the table has to be read as a ranking of which model judges most like those two rather than a ranking of which model is right.

The question of whether a student can pass its teacher follows close behind. A strong student trained on labels from a weaker teacher outrunning that teacher has been reported since 2023 under the name weak-to-strong generalisation. Follow-up work through 2026 attaches conditions. Research on distillation scaling finds that the excess performance appears only while training data is limited, and that with enough data the student's loss rises again and converges on the teacher's level. The related point, that treating the teacher's policy as the imitation target makes the teacher's limit the student's ceiling, runs the same way.

Whether 67.8% is the model's ceiling or the answer key's ceiling cannot be settled from what the company has published. Settling it would take re-scoring the same four tasks against answers assigned by people. That experiment has not been published by anyone.

One claim sits ahead of price in the company's own ordering: every answer comes back with a probability. Keep that promise and the calling code gets shorter. Above a confidence of 0.95, process automatically; below it, route to a human queue. Cutting human review from everything to a subset becomes a single threshold, and for an organisation moving judgment work this is where the value is, more than in the unit price.

So this section asks one thing. Returning a probability and returning a probability that holds are separate problems. How far does this model's probability hold?

4.1There Is Already a Name for a Probability That Holds

The standard measure is calibration error. Gather the items answered with a confidence of 0.9, count how many were actually right, repeat across the confidence range, and average the discrepancy. If 90% of the 0.9 items are correct, the model is well calibrated; if only 70% are, it is overconfident. Three strands of prior work already cover how such probabilities get used.

• Selective prediction. The system abstains on low-confidence items and answers the rest. Abstain more and the error rate among answered items falls while the share of traffic handled shrinks. The curve tracing that trade-off is the risk-coverage curve, standard equipment in deep learning work since 2017.
• Deferral to a human. Instead of discarding the abstained items, the system sends them to a person or another expert. Work in 2018 and 2020 formalised learning the deferral rule jointly with the classifier. When an organisation says it will send everything below a threshold to a human, that is the academic name for the design it has chosen.
• Conformal prediction. Rather than one answer, the system returns a set of answers that contains the true one with a stated probability. Its strength is a guarantee that assumes nothing about the distribution; the price is that the guarantee holds only while new data remains exchangeable with the data used to calibrate it.

A small irony lives here. The Arena chief executive from section 1, the one who could not see what separates this from a standard zero-shot classifier, wrote the standard introduction to conformal prediction. Scepticism from the part of the field that has spent years formalising uncertainty with guarantees attached carries a slightly different weight. Returning a probability is not new. Establishing when that probability holds would be.

When probabilities break is also textbook material. A large 2019 comparison study established across many methods that as the input distribution drifts away from the training distribution, accuracy degrades while confidence does not degrade nearly as fast, so calibration error widens sharply. Whether that pattern reproduces on Jev is the question the four independent evaluations below answer.

4.2Four Evaluations the Vendor Did Not Design

Within ten days of launch four separate parties tested the model their own way, and all four published data and code. Unlike the vendor table in section 3, none of these evaluations was designed by the company being evaluated.

All four publish their source repositories and reproduction code. The figures in the discussion below are taken from those originals.

The first of them came closest to backing the company's claims on speed and price. The evaluator put 21 style questions to each of 27 of their own essays and 10 deliberately machine-sounding controls; 777 judgments came back inside 0.7 seconds for a little over a quarter of a cent. A separate experiment in the same piece took 12 synthetic paragraphs, seeded six of them with defects, and ran Jev beside Fable 5.1. Of the 7 seeded defects Fable caught all 7 and Jev caught 6. Median processing time per paragraph was 0.35 seconds against 8.83, and the cost ratio came to roughly 580x. The character of the single miss matters more than the count. What Jev missed was the same kind of thing every time: passages where a described action is never given a reason, the example being a shared calendar supposedly taught jointly by parents and staff, with the connective tissue absent. That is less a random dropped item than a blind spot for one category. Over the early-access period this evaluator ran 11 experiments and 1,709 judgments for under a cent in total, and their own conclusion was that it works well as an early-warning device but that they would want a more thorough accuracy check before putting it in production.

The second evaluation registered 50 predictions with timestamps before collecting any data. The design commits the authors to publishing their own wrong predictions, and 26 were confirmed while 21 were falsified. They also recorded that 18 of those 21 failed in the same direction, and the direction is this one: the model kept succeeding where the team had predicted failure. A group that had tied its own hands with pre-registration was systematically underrating the model, which is worth holding onto, because the critical numbers quoted below come from those same evaluators. This is where the 95.9% on the 400-item benchmark comes from. The same study also produced the heaviest number in this report.

The third evaluation went straight at the probabilities. It ran 900 rule-generated synthetic tickets the model could not have seen alongside 3,721 items from public benchmarks. On the public benchmark slice, calibration error came in between 0.024 and 0.032, which is close to unproblematic. On the synthetic slice it was 0.107, which is 4.4 times the noise floor of 0.024 measured in the same experiment. Inside the distribution the probabilities are honest; outside it they collapse. That is the 2019 pattern, reproduced.

4.3Split the Question and the Same Model Changes Sides

The fourth evaluation aims not at accuracy but at what accuracy depends on. Its starting point is a line of methodology the company put in its own blog.

"The most reliable real-world workflows tend to have many independent, decomposed questions, with fine-grained behavior that's dependent on probabilities instead of discrete decisions."

TypeSafe's official blog. The sentence sits in the passage explaining how the evaluation was designed.

A phishing email benchmark published on 17 September put that sentence to the test. It drew 2,000 emails from a public dataset built with synthetic bodies, half phishing and half legitimate, and asked the same model in two ways. Once as a single question: is this email phishing? Once as five separate questions — does the sender domain diverge from the link domain, is free hosting in use, is the reader pushed to confirm an account or log in, is there time pressure, is the sender address an ordinary one — with the five answers combined by logistic regression.

The single-question figures cover all 2,000 emails; the decomposed figures were measured on the 1,000 held out from fitting the regression. Area under the curve for the single question was 0.689 for Jev against 0.837 for Haiku; after decomposition it was 0.982 against 0.991, with Haiku now narrowly ahead. On cost per 1,000 emails, one Jev judgment ran $0.038 and asking all five signals still ran $0.04, while Haiku on five signals came to $1.02.

Same model, same emails, same day. The only thing that changed is how many questions the work was cut into, and the conclusion moves from "well behind" to "hard to tell apart." That result and the 16.7% in 4.2 are two faces of one fact. Get the wording wrong and the model does worse than guessing; cut the wording into the right pieces and it holds its own against a frontier model. Accuracy is not a property of the model so much as a value produced by the pairing of a model with a question design, and the 67.8% from section 3 cannot be read apart from how that evaluation harness posed its questions either.

A practitioner about to copy this result into a business case should look one layer further into the design. What combined the five answers was not the model but a logistic regression fitted on 1,000 emails. Decomposition pays only if you have labels to fit the combining layer with, and producing those labels is the real cost of the method. Beyond that, the ground truth in this benchmark did not come from people reading emails; it came from link reputation data, and the bodies are synthetic. Whether that adds up to sufficient evidence for moving work to the cheaper option is a call to make with those conditions in view.

4.4It Cannot Say That It Does Not Know

Back in the out-of-distribution study, the most operationally useful result is the slice where the answer is not in the ticket at all. These are questions that require internal company policy, so no model could get them right by any route. Accuracy there was 44.7%, and the mean confidence the model attached was 0.74. Rather than signalling that it did not know, it produced fairly confident wrong answers.

The out-of-distribution study measured the direction of the miscalibration as well, and that direction lines up with the three primitives from section 1. Choice and Score skew toward overconfidence; Noul skews toward underconfidence. The evidence is the temperature coefficient needed to rescale the returned probabilities: 1.30 for Choice and 1.92 for Score, against 0.66 for Noul, which is below 1. On the unanswerable set the coefficient climbed to 3.40. A miscalibration whose sign changes with the type of question means a design with one threshold for the whole model does not hold together.

What happens if you set the threshold very high was measured by the pre-registered study. Translated into the language of risk and coverage, the results look like this.

The first four rows come from the pre-registered study; the last is from the out-of-distribution work, measured on its synthetic slice. That last row says the separate confidence value the model supplies was less honest than the largest of the option probabilities. Two studies measured different things on different data and arrived at the same recommendation.

The out-of-scope row is the centre of that table. A high threshold filters well for what the model knows and does not filter at all for what it does not. The high-confidence band is clean because the easy items collect there, not because the dangerous items were pushed out of it. All thirty genuinely out-of-scope inputs passed through on whichever side of the threshold they landed. The example the report gives is a cake recipe. Belonging to none of the categories on offer, it drew a confidence of 0.94.

Locating the cause in the model alone leaves nothing to fix. That experiment's option list contained no "none of these." The property from section 1, that the only things available to pick are inside the list, works in reverse here. Given an input with no right answer, the model picks the least wrong cell it has and then attaches a probability to that pick. Which is why the two recommendations the pre-registered study leaves behind are about the calling code rather than the model: gate at 0.99 or not at all, and put an explicit none-of-these option into every Choice.

The calibration study's recommendations point to the same place. Calibrate per question rather than per model; do not put a threshold on the separate confidence field the model returns; and treat the returned probabilities as scores whose ordering you can trust rather than as probabilities, refitting them on your own data. One balancing observation from the same study belongs here too. The size of the overconfidence observed out of distribution is comparable to what a fine-tuned classifier shows in the same situation. This model is not uniquely bad; probabilities collapsing on inputs outside the distribution is a general property of the family.

Which sends us back to the 0.95 threshold this section opened with. That one line does make for short code, but as measured, wherever you put the threshold between 0.50 and 0.95 the accuracy above it does not change. The one band that justifies automatic processing is above 0.99, and that band is 60.2% of traffic. The promise of a probability alongside the answer is not a promise that the probability is always honest. It is the much narrower promise that it can be honest in known situations. The distinction matters because the first thing an organisation moving judgment work wants to cut is human review, and the thing it wants to cut it on is exactly this probability. What separates the reducible band from the rest is not the height of the threshold but whether the input falls inside the training distribution. Very few pipelines currently in production measure that.

Gather what has been measured so far and it comes to this. The price is cheaper by anything from five times to several hundred times depending on the comparison; the accuracy trails by 2.3 to 17.3 points depending on the task; the probabilities are honest only inside the distribution. With those three in hand, the question a practitioner actually has to answer is single: of the judgment calls already wired into our pipeline, which ones can move?

Starting from price sends you off course. What to look at first is the shape of the damage a single wrong answer does. The table below breaks that judgment into five questions, and the more answers land in the left column, the more safely the work can move.

A decision table for moving judgment work. The first three questions are about operational design; the last two are closer to conditions that block the move outright. As section 4 showed, the final row is not something the model's own probability can decide for you.

Spam classification, ticket routing, log anomaly triage and document tagging mostly gather in the left column. An error can be fixed by the next person, the same input can be rerun, and low-confidence items can pile up for a human to sweep through. Loan denial, rejection of a job application, refusal of insurance cover and permanent account suspension fall to the right on all four. One judgment is the notification, and the person who receives it already has a legal right to ask why.

5.1The Stricter Regime Right Now Is Korea, Not Europe

When coverage of this launch reaches for a regulation, it usually reaches for the EU AI Act. Check the tense of both regimes as of September 2026, though, and the order is the reverse of what that implies. Europe has not arrived yet. Korea already has.

Take Europe first. Annex III of the AI Act classifies systems that evaluate the creditworthiness of natural persons or establish their credit score as high-risk. Financial fraud detection is carved out, and corporate credit models applied to legal persons fall outside the provision as well. The application date for those high-risk duties, however, was pushed to 2 December 2027 by an amending regulation that entered into force in July 2026, on the grounds that the standards and conformity-assessment machinery would not be ready in time. The penalty tier is frequently misquoted too: the ceiling attached to high-risk violations is €15 million or 3% of worldwide turnover, whichever is higher. The €35 million and 7% figures belong to a different tier, the one for prohibited practices.

Deferred application and light obligations are not the same thing. Two of the provisions that attach on a high-risk classification bear directly on the subject here. Article 10 requires governance over training, validation and testing data, and its list covers how the data was collected, what it was originally collected for, preparation work such as annotation, labelling and cleaning, and records of examination for bias. The same article requires the data to be relevant, representative and, to the best extent possible, free of errors and complete. Article 12 requires the system to be capable of automatically recording events over its lifetime, so that hazardous situations and substantial modifications can be identified and post-market and operational monitoring is possible.

Article 12 comes with a qualification. The part of it that pins down what must be logged and in what detail is paragraph 3, and paragraph 3 attaches to the biometric category under Annex III point 1(a). What applies to credit scoring is the general duty in paragraphs 1 and 2, where the items are not specified. What to retain is therefore still the operator's call, and that space is exactly where section 6 comes back to. What the law asks of an organisation that multiplied its judgments a hundredfold is not evidence that the price was low but the record it kept.

Korea runs the other way. The AI Framework Act has been in force since 22 January 2026, and loan screening and hiring are classified as high-impact AI. Operators have to stand up a risk management system, have to be able to explain why a given person was rejected, and have to test for bias by gender, age or region. The responsible ministry has said it will allow a grace period of at least a year, and there is a misunderstanding attached to that.

An older set of rules overlaps on top. Article 36-2 of the Credit Information Act has guaranteed three rights over automated evaluation for several years now: the right to be told whether an automated evaluation took place and to have the result and its main criteria explained, the right to submit favourable information or demand correction of the underlying data, and the right to demand recalculation. The enforcement decree spells out how credit grades and scores must be presented and what information was reflected, and to what degree.

5.2A Human Making the Final Call Is Not an Exit

It is easy to read this product's design as a way around the rules. The model only returns a probability, a person makes the final call, so this is not automated decision-making. The European text closes that route. A system that is not the sole decision-maker still falls under the high-risk classification if it produces a score or a recommendation and a person acts on it. A separate provision gives the person affected by the decision the right to a clear and meaningful explanation of the role the AI played in it.

Korea points the same way. The duty to explain attaches on the basis of whether an automated evaluation fed into the result, not on whether the final decision-maker was human. Any organisation planning to use this model in a high-impact area therefore needs to prepare not a case that the price is low but a record, per judgment, of what the grade rested on. As it stands the model returns an answer and a probability, and no sentence of reasoning.

Finance carries another layer. Korea's Financial Services Commission consolidated three previously separate sets of financial-sector AI guidance into one document, effective 22 June 2026, and one of its seven principles is that AI serves as an auxiliary means. At the current stage, the text reads, AI is an aid to the work, so the final decision and the accountability that comes with it rest with staff. Keeping a human as the final decision-maker, in other words, is closer to a requirement than to a shield. The guideline is a voluntary code of best practice rather than a legally binding regulation, but the same document states that where a system qualifies as high-impact AI, separate legal duties arise under the AI Framework Act and its subordinate legislation. Its scope is not narrow either: it covers fintech firms whose AI outputs can affect financial transactions as well as financial companies, and it reaches internal support tools rather than only customer-facing services. The Financial Supervisory Service's risk management framework and the Financial Security Institute's security guide were issued the same day.

Early industry usage tracks that line. Several outlets carried the same assessment from Atlassian's chief product officer, Tamar Yehoshua, that the model is far cheaper and far faster but does not match the leading models on accuracy, along with the detail that the company is testing it on classification work only and not for long-form writing. Which outlet ran that first, we could not establish. The tenor is closer to a gain in places where a model was previously unusable because it cost too much and ran too slowly. Not moving existing work because the price dropped, but starting work the price had made impossible.

The model is named after a person: William Stanley Jevons, the nineteenth-century British economist. The argument of his 1865 book The Coal Question still lands. Against the common assumption that better coal efficiency in steam engines would reduce coal consumption, he argued that improved efficiency opens up more places to use the thing, so total consumption rises instead. A resource that gets cheaper is not used sparingly; it spreads into more slots.

The company has been open about why it chose the name. Its founder has said the name carries the expectation that cheap machine intelligence will lead to far broader deployment. At a cent per judgment you make ten thousand judgments a day. At a hundredth of that you make a million. What falls is the price per call; what rises is the number of calls.

6.1Models Are Already Crowding Into the Same Slot

The proposition that an efficiency gain draws a crowd shows up first on the supply side. The Batch names three competing models in the same article: Raya, claiming higher accuracy and speed on multilingual work; Bespoke Nimble, a fine-tune of an open model; and Kev, an attempt to reverse-engineer Jev's structure. Ten days after launch. What those last two stand on is the substance of the point. Bespoke Nimble is the 9-billion-parameter release of the open-weights Qwen 3.5 fine-tuned for classification, and Kev takes the same family as its floor and ships in three sizes. Neither involves training a new model from scratch; both adapt something already downloadable. That is why the cost of entering this slot is low.

Demand-side signals exist too, and this is where reading carefully matters. Vercel, which runs a model gateway, wrote that within 24 hours of launch close to 13% of its paid teams had used the model, and that it reached one team in ten within 18 hours. It added a comparison: more than twice the rate of the GPT-5.6 family on the same platform, and more than six times that of Fable 5.1. But the figure has no denominator. How many paid teams there are in total is published nowhere, and more importantly the model was free on that gateway until 25 September. The 24-hour window in which the 13% was observed overlaps the free period.

6.2Getting the Tense Right on the Valuation Story

The sentence attached most often to this story is that the valuation jumped from $200 million to the $10 billion range in a single week. Follow the primary sources and two parts of it come apart.

Correct both statements and the nine-day span stops existing. The first valuation was set more than a year ago; the second has not been set at all.

That does not make the discussion groundless. James Hardiman of DCVC, which led the seed, has said inference costs are so low that the company is already profitable. That is an investor speaking rather than a financial disclosure, so it is not a fact to copy across, but it does suggest that a figure in the $10 billion range is attached to a cost structure instead of to a revenue projection. A model with nothing billable on output changes the income statement of the company selling it too.

The correction may look minor, but it is the same work that has been going on since section 2. The numbers are usually not wrong. What drops away, as they are copied from document to document, are the conditions that were attached to them. The qualifier "high end" drops. The comparison model drops. The tense "under discussion" drops. Putting those conditions back is the most practical thing a report like this can do.

6.3What Multiplies Is Not the Judgments but the Records of Them

Move Jevons's argument into this market and one more thing follows. Multiply judgments a hundredfold and the records of those judgments multiply a hundredfold as well. Each record holds what was asked, what the options were, which answer came back, and how confident it was. The pile has two faces. On one side it is the first judgment history an organisation has ever owned. On the other it is a heap of labels nobody has checked.

Full-population inspection becomes possible from here too. If sampling was how data quality checks ran because checking everything cost too much, $0.0004 per call removes the constraint. As section 4 established, though, the honesty of those judgments holds only while the input sits inside the training distribution. The moment full inspection becomes affordable, the range of inputs entering the pipeline widens as well, so the very event that lowered the price raises the share of out-of-distribution inputs. The judgments added because the price fell are the ones most likely to be the least trustworthy.

The real question raised by judgment costs falling to a hundredth is therefore not how much was saved. It is who checks the quality of the enlarged pile of judgment records, with what, and when. That question overlaps exactly with the ground Pebblous has been working for years.

Much of what Pebblous does in DataClinic has the same shape as what this model does: pointing at data and judging what it is, what it scores, whether it passes. So this launch is not competitor news for us. It is news that the unit price of the process we stand in has changed. What sections 2 through 6 measured is also what stops us from reading that price straight off as an opportunity. A few things need separating.

7.1Sampling Can Become Full Coverage, but the Arithmetic Has to Be Redone

If cost was the reason data quality checks ran on samples, that reason genuinely disappears. At $0.0004 per call, sweeping a ten-million-row table end to end costs four thousand dollars, and that is the range where things previously invisible because nobody could look at all of them start to show up. It also means the effort Pebblous puts into sampling design when handling customer data can move over to inspection design.

What section 2 demonstrated, though, is that what the price actually is ranges from five times to 440 times depending on the comparison. The useful end of that range for practical work is the bottom. Against the model closest to Jev on accuracy inside the vendor's own table, the per-call difference is eight times. An organisation already running its judgments through a cheap small model gains that eight times by switching, not several hundred, and whether sampling can become full coverage usually turns on exactly that single-digit multiple. The arithmetic for switching to full coverage has to be redone with the denominator from your own pipeline in place of the one from a vendor headline. Which model runs that judgment today, what it costs per call, and what share of those calls was picking rather than writing. Until those three are measured, 440x means nothing.

7.2Who Labelled This Appears Twice in the Product Spec

It is hard to find a place in this product where a person attached a label. The evaluation's answer key is the average of two larger models, and the company did not conceal it — it wrote it down. The training side comes from a different grade of source: the statement that the model was trained entirely on synthetic data is TechCrunch's sentence from 18 September, and the company has never published its training composition. Either way, this product is a case of the question AI-Ready Data has been asking all along arriving inside a product spec. Who attached this label, and by what criteria?

Leave the question unanswered and one thing stays unresolved. As section 3 set out, whether 67.8% is the model's ceiling or the answer key's ceiling cannot be known until the same tasks are re-scored against answers assigned by people. This is a provenance problem rather than a benchmark-score problem. Where the origin of a label goes unrecorded, an argument about performance cannot reach a conclusion. It is the same ground Pebblous covered in Every Click on the Labeling Screen Becomes Audit Evidence.

That this is not one vendor's situation is what the phishing benchmark in 4.3 shows. The ground truth in that evaluation, mounted to check the vendor, did not come from people reading emails either; it came from link reputation data. The labels on the checking side face the same question, and we can ask it only because the authors wrote the condition into their repository. What Article 10 of the EU AI Act asks of high-risk systems, as section 5 described, is precisely that record: where the data was collected, what it was collected for, how the labels were attached, and how bias was examined. That the items regulation is belatedly asking for and the gap this launch exposed are the same list is not a coincidence.

Whether a model trained only on synthetic data can be trusted hangs on this as well. The widely cited 2024 result that repeated training on generated data collapses the distribution has a counterpart pointing the other way: keep the real data and accumulate rather than replace, and the collapse does not occur. The two studies measured different conditions, and which one applies to this product cannot be judged while the training composition stays unpublished. Not filling in the unpublished parts by guesswork is this report's rule.

7.3The First Job Is to Count the Calls Separately

Splitting the model calls in a customer pipeline into generation work and judgment work, and counting each, has become work that pays immediately. Almost no organisation has measured that ratio, and no public figure representing the industry exists. Several accounts of routing architectures put easy, judgment-shaped work at more than half of traffic, but those are individual companies talking about their own traffic, not an industry statistic. The two should not be mixed.

An organisation measuring that ratio now has no average to copy; it is taking its own figure for the first time. What to consult after the measurement is the decision table in section 5. The unit of judgment is the task rather than the model, and the criteria are the asymmetry of the cost of a wrong answer, whether a retry is possible, whether there is a path to a human, and whether the law requires an explanation.

Where Pebblous stands in this, finally. When judgments get cheap, judgments multiply, and the enlarged pile of judgment records is both a new data asset and a new quality debt. Regulation lands on top of that. In Korea, as of today, loan-screening AI is already high-impact and already subject to duties of explanation and bias testing. Our position is not on the side selling the model but on the side that measures whether those judgment records can be trusted and keeps the evidence. An event that cut the price to a hundredth does not shrink that position. It widens it a hundredfold.

Sources for the figures in this report, grouped by what kind of source they are. Items 1 to 3 are material the company published itself, and every price and accuracy figure comes from there. Items 7 to 11 are evaluations the vendor did not design, with data and code in the open. Items 12 to 18 are the statutes and administrative guidance this report read directly. The academic literature from item 19 onward is not research about this product; it is general work that can confirm or refute the product's claims. No paper about Jev itself exists.

Vendor primary sources

• 1.TypeSafe AI, "Introducing System One Models & Jev" (Diogo Almeida, 15 September 2026). typesafe.ai — the source of the three sentences quoted in sections 2, 3 and 4.3 (the higher end of real-world gains, the answer-key bias, and the in-house construction of the evaluation tasks). The three output primitives, the expansion of RLCD, the $0.042 input price and the unbilled output line were confirmed here as well. Accessed directly on 27 September, the page metadata returned 26 September as the last-modified time, and whether the bias concession was in the original version could not be confirmed.
• 2.TypeSafe workflow evaluation dashboard. evals.typesafe.ai — the aggregate table in section 2 and the per-task accuracy table in section 3 come from here. This report verified that averaging the per-task values reproduces the published aggregates. The absolute item counts behind the four tasks are not in this material.
• 3.TypeSafe home page (pricing and headline claims). typesafe.ai — where "193.6x Faster, 444.6x Cheaper" hangs, along with the 238x input-price multiple, stated there as against Claude Fable 5.1. Neither headline multiple names a comparison model on this page either.

Coverage

• 4.The Batch, issue 372 (25 September 2026). deeplearning.ai — the per-call figures of $0.0007, $0.06 and $0.12, the rounded 67% total, and the names of the three competing models come from here. That this article's per-call values diverge from the company dashboard by different amounts on different lines is the subject of section 2.3.
• 5.AI Times (Lim Dae-jun, 26 September 2026). aitimes.com — Korean coverage that re-confirms the pricing and the scepticism. The "one four-hundredth" and "up to 200x" in that article appear to be rounded restatements of the company's own claims, and the article does not say which model either is measured against.
• 6.Financial Times (Nicole Jeffrey, 25 September 2026) and The Information (24 September 2026) — the source of the Arena chief executive's verbatim scepticism, the founder's account of when the seed closed, and the $1 billion raise under discussion. The Financial Times original is paywalled and this report verified it through a mirror. Because quote boundaries can shift in mirrored copies, the remark appears in the body only with its attribution clause intact.

Independent evaluations (not designed by the vendor)

• 7.Mike Taylor (Every), "Mini Vibe Check: TypeSafe's Jev…". every.to — 21 questions across 37 documents for 777 judgments, under 0.7 seconds, about a quarter of a cent. The separate experiment in the same piece supplies the median of 0.35 seconds against 8.83 per paragraph, 6 of 7 seeded defects, and the roughly 580x cost ratio.
• 8.PriorBench, pre-registered evaluation (20 September 2026). github.com/priorbench/jev — of 50 pre-registered predictions, 26 confirmed and 21 falsified, 18 of those in the same direction (failure predicted, success observed); 21 experiments, 5,721 calls, $0.176 in total. The 400-item zero-shot 95.9%, the 77.2% and 66.0% controls, and the 16.7% under wrong question wording are this study's results. The first four rows of the table in 4.4 (100% accuracy above 0.99, the flat band from 0.50 to 0.95, 60.2% coverage, 0 of 30 out-of-scope items caught) are also from this study — the repository's own phrasing is "Gate at 0.99 or not at all" and "Always offer an explicit 'none of these' option." The version tested was jev-1.13-20260917, and the author notes the limitation that measurement went through a gateway, so model latency and gateway latency (a fixed component of about 430 milliseconds) are not separable.
• 9.scienthoon, out-of-distribution calibration study (19 September 2026, MIT licence). github.com/scienthoon/jev-ood-calibration — 900 synthetic tickets (300 each of Choice, Score and Noul) plus 3,721 public benchmark items, at a reproduction cost of about $0.06. The synthetic-slice calibration error of 0.107 against a noise floor of 0.024, the public benchmark range of 0.024 to 0.032, the 44.7% and mean confidence of 0.74 on unanswerable questions, and the confidence field's calibration error of 0.18 on the synthetic slice all come from this study. So does the split in temperature coefficients — 1.30 for Choice, 1.92 for Score, 0.66 for Noul — which is why the sign of the miscalibration differs by question type.
• 10.jev-phishing-bench, phishing email classification benchmark (17 September 2026, MIT licence). github.com/anisselbd/jev-phishing-bench — 2,000 emails from the public PhishNChips v5.2 dataset (1,000 phishing, 1,000 legitimate). Asked as one question, Jev 62.6% and Haiku 4.5 81.3%; decomposed into five signals and combined by logistic regression, Jev 95.0% and Haiku 93.2% on the held-out 1,000. Areas under the curve, costs and latencies are in the repository, as are the conditions that the ground-truth labels came from link reputation data and the bodies are synthetic.
• 11.Vercel, "AI Gateway: Jev model launch" (18 September 2026). vercel.com — the source for 13% of paid teams within 24 hours and 10% within 18 hours. The denominator is not published, and the figure has to be read alongside the fact that the model was free on that gateway until 25 September.

Statutes and administrative guidance (read directly)

• 12.EU AI Act, Annex III point 5(b). artificialintelligenceact.eu — classifies the evaluation of the creditworthiness of natural persons and the establishment of their credit score as high-risk, and excludes financial fraud detection. The reading that a system escapes nothing by not being the sole decision-maker rests on this family of provisions.
• 13.Regulation (EU) 2026/1744 (the Digital Omnibus) — published in the Official Journal on 24 July 2026 and in force from 27 July. It moved the core obligations for Annex III high-risk systems to 2 December 2027 and the Annex I family to 2 August 2028. The original date was 2 August 2026, so the deferral was settled six days before the deadline.
• 14.EU AI Act, Article 99(4) (penalties) and Article 86 (right to explanation). artificialintelligenceact.eu — the ceiling for breaching high-risk obligations is €15 million or 3% of worldwide annual turnover, whichever is higher. The widely quoted €35 million and 7% belong to a different tier, for prohibited practices.
• 15.EU AI Act, Article 10 (data and data governance) and Article 12 (record-keeping). Article 10 · Article 12 — the basis for section 5.1. Article 10(2) enumerates collection provenance, original collection purpose, preparation work such as annotation, labelling and cleaning, and records of bias examination as governance items, and 10(3) requires relevance, representativeness and, to the best extent possible, freedom from error and completeness. Article 12(1) requires automatic logging capability over the lifetime and 12(2) requires those logs to support identification of hazardous situations and post-market monitoring. Paragraph 3, which specifies logging items, attaches to the Annex III point 1(a) family and so does not apply as written to credit scoring.
• 16.Korea's Framework Act on AI Development and Establishment of a Foundation for Trustworthiness — promulgated 21 January 2025, in force 22 January 2026. The duties of operators of high-impact AI, including loan screening and hiring, sit in this act. The grace period is the responsible ministry's administrative policy rather than a statutory deferral, and the supplementary provisions contain no clause postponing application. A partial amendment of 20 January 2026 (Act No. 21311) took effect on 21 July, so the current version should be checked when citing specific articles.
• 17.Credit Information Act, Article 36-2 and Enforcement Decree Article 31-2 — the right to an explanation of an automated evaluation result, the right to demand correction or deletion of the underlying information, and the right to demand recalculation. The Financial Services Commission's validation framework for AI-based credit scoring models also treats the feasibility of meeting the explanation duty as a check item.
• 18.Financial Services Commission, revised Guidelines on AI in the Financial Sector — fsc.go.kr. Effective 22 June 2026. The seven principles cited in 5.2 (governance, legality, auxiliary means, reliability, financial stability, good faith, security) and the wording of the auxiliary-means provision come from here. It is a voluntary code of best practice rather than a legally binding regulation, and the same document states that separate duties under the AI Framework Act and its subordinate legislation arise for high-impact AI. The Financial Supervisory Service's AI risk management framework and the Financial Security Institute's AI security guide were distributed on the same date.

Academic literature (general work that tests the product's claims)

• 19.Lianmin Zheng et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." NeurIPS 2023 Datasets & Benchmarks. arXiv: 2306.05685 — the naming of position, verbosity and self-enhancement bias in section 3.3 comes from here.
• 20.Koki Wataoka et al. "Self-Preference Bias in LLM-as-a-Judge." arXiv: 2410.21819, and "LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods" arXiv: 2412.05579 — the measurement of self-preference bias, the difficulty of separating it from genuine quality, and the expansion of the bias list are the content of these two.
• 21.Collin Burns et al. "Weak-to-Strong Generalization" (2023) and "Distillation Scaling Laws" arXiv: 2502.08606 — the distillation-ceiling discussion in section 3.3. The second reports that the window in which a student surpasses its teacher is confined to limited training data, and that more data converges the student on the teacher.
• 22.Yaniv Ovadia et al. "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift." NeurIPS 2019. arXiv: 1906.02530 — the basis for section 4.1. It established across many methods that as the distribution shifts, accuracy falls while confidence does not fall with it, widening calibration error.
• 23.Yonatan Geifman, Ran El-Yaniv. "Selective Classification for Deep Neural Networks." NeurIPS 2017 — the source of the risk-coverage curve. David Madras et al. (2018) and Hussein Mozannar, David Sontag (ICML 2020) formalised the design that routes abstained items to a human.
• 24.Anastasios N. Angelopoulos, Stephen Bates. "Conformal Prediction: A Gentle Introduction." Foundations and Trends in Machine Learning 16(4), 2023. arXiv: 2107.07511 — the standard introduction to distribution-free uncertainty quantification referred to in 4.1, written by the same person quoted in section 1.
• 25.Ilia Shumailov et al. "AI models collapse when trained on recursively generated data." Nature 631, 755–759 (2024). doi:10.1038/s41586-024-07566-y, and Matthias Gerstgrasser et al. "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data." arXiv: 2404.01413 — the two strands on synthetic-only training in 7.2. The first examines replacing real data with generated data; the second examines accumulating rather than replacing.
• 26.William Stanley Jevons. The Coal Question (1865) — the starting point of section 6. The classic argument against the assumption that efficiency reduces consumption, holding that total consumption rises instead.

Adjacent Pebblous writing

• 27.Every Click on the Labeling Screen Becomes Audit Evidence — referenced in 7.2. It approaches the problem of recording who attached a label and on what criteria from the regulatory side.