How much does one grading decision cost with Jev compared with a language model?

In LangChain's published test, 500 decisions cost $0.34 with Jev and $28.17 with Claude Sonnet 4.6, which divides to $0.00068 and $0.05634 per decision; LangChain also reports Jev's average at $0.00035 per call. On Openlayer's benchmark, 1,000 samples cost $0.03 with Jev, $0.46 with one call to GPT-4.1-mini, and $2.60 through Ragas on the same model. Both are third-party measurements on their own workloads, at list prices.

Why is the price per call the wrong number to approve?

Because grading is charged per decision and decisions multiply: every check on every change, every rerun, every message through a guardrail. A price of five cents per call reads as nothing until 800 checks a day turn it into $1,352 in a 30-day month. Approve the monthly figure at your own volume, computed from the per-decision price, and the decision becomes obvious in either direction.

Is TypeSafe's claim that Jev is 444.6 times cheaper reliable?

Treat it as a vendor claim with two stated caveats. TypeSafe's own capabilities team wrote the workflows behind the comparison, which finished in 0.114 seconds for $0.000081 against 8.566 seconds for $0.013880 with GPT-5.6 Terra, and TypeSafe says it cannot prove its price is unsubsidised. The third-party figures from LangChain and Openlayer are smaller ratios on independent workloads, so budget from those.

Where does the saving in Openlayer's benchmark come from?

Two places, and the middle row of the table separates them. Going from $2.60 to $0.46 per 1,000 samples came from asking one structured request per sample instead of six language-model calls plus embeddings, on the same GPT-4.1-mini, and that is a design change you can make with any model. Going from $0.46 to $0.03 came from Jev's price, $0.042 per million input tokens with no charge for output.

When is moving a grader to Jev not worth it?

When the volume is low, when you need a written reason, or when the checks fall on Jev's known weaknesses. At 20 decisions a day the language-model judge costs $33.80 in a 30-day month at LangChain's per-decision price, and labelling a calibration set costs more than a year of that. A check that needs a paragraph for an auditor, or that involves counting or dates, should stay with a language model or a person.

What costs does the Jev price list leave out?

Four. A calibration set your team labels by hand before any Jev grade may fail a change. A fallback path for the 529 overloaded and 429 rate-limit errors TypeSafe documents. Version pinning, because jev-latest moves and every model version means rerunning the labelled set. And the cost of depending on a vendor whose model was announced on 15 September 2026 and left its waitlist on 20 September.

Does a faster grader change the eval budget beyond the price per call?

Yes, because a suite that waits on its grader costs engineer time on every run. On our own suite, with Jev as the grader, the run is ten times faster than it was with the previous language model. LangChain measured 0.44 seconds per Jev call against 2.16 to 2.83 seconds for the language models it compared, and that difference is paid on every check.

Should guardrails on every message use Jev?

At guardrail volume the arithmetic favours it: one million messages a month cost $2,600 through a six-call Ragas pipeline, $460 through one call to GPT-4.1-mini, and $30 through Jev at Openlayer's per-sample prices. TypeSafe's guardrail cookbook shows the policy shape, with a review threshold at 0.35 and an action threshold at 0.70 or 0.85. Anything that moves money still goes to a person regardless of the grade.

How do you budget for a vendor that left its waitlist one week ago?

Name the exact model version in your configuration, jev-1.13.0 at the time of writing, and keep the previous grader wired as the fallback so a price change or an outage costs you a switch rather than a stopped suite. Budget the calibration set as a one-time cost and a rerun of it per version. Assume the list price can change, because the vendor says it cannot prove it is unsubsidised.

How does grading cost differ between evals and guardrails?

An eval suite pays per change, multiplied by the checks per change and the reruns, so volume follows how often engineers commit. A guardrail pays per message, so volume follows how many people use the product. Evals grow with the team and guardrails grow with the customers, and the second grows faster, which is why guardrail budgets are where the per-decision price matters most.

The Real Price of an LLM Judge: Per Decision, Multiplied: Reveneau

A line in an eval budget reads like a small number. A language model used as the judge, the grader that decides whether a piece of AI-written code matched its acceptance criterion, costs a few cents per call. Somebody approves the few cents, and the invoice at the end of the quarter surprises them.

The mistake is approving a price per call. Grading is charged per decision, and decisions multiply: every check on every change, every rerun after a fix, every message that passes through a guardrail. Here is the arithmetic from public figures only, when the saving from a cheaper decision model pays for a migration, when it does not, and the four costs on the Jev side that the price list leaves out.

1. The published prices, per decision

Jev, the decision model TypeSafe AI released on 15 September 2026, is priced at $0.042 per million input tokens, and output tokens are free. Tokens are the pieces of text a model reads, and billing counts them. That price alone tells you little, because a decision is not a token. Two third parties have published what a decision costs on their own workloads.

LangChain's test, 20 September 2026, ran five agent cases 100 times per judge, 500 decisions each. The full run cost $0.34 with Jev and $28.17 with Claude Sonnet 4.6. Divide each total by 500 and a decision costs $0.00068 with Jev and $0.05634 with the language model. LangChain also reports Jev's average at $0.00035 per call, a lower figure than the run total divided by its decisions, and the post does not say what else the total includes, so the figures below use the totals. LangChain's authors call their results observational and note that low cost can amplify mistakes, and five cases is a small set.

Openlayer's jevals benchmark gives three rows for the same evaluation of 20 samples. Ragas, an evaluation library, running on GPT-4.1-mini, used 6 language-model calls per sample, plus embedding calls that turn text into numbers for comparison, and cost $2.60 per 1,000 samples in 22 to 35 seconds. The same GPT-4.1-mini asked one structured request per sample cost $0.46 per 1,000 in 4 seconds. Jev, one request per sample, cost $0.03 per 1,000 in 0.8 seconds. Openlayer says the language-model costs are at list price and that Jev's cost is input tokens only. Per sample: $0.0026, $0.00046 and $0.00003.

The middle row is the one that matters most. Most of the gap between $2.60 and $0.46 came from asking one question per sample instead of six calls, on the same model. That is a design change you can make with any grader. The gap between $0.46 and $0.03 is Jev's price.

Then there is the vendor's own number. TypeSafe reports a workflow decision finishing in 0.114 seconds for $0.000081 on Jev against 8.566 seconds for $0.013880 on GPT-5.6 Terra, which it presents as 193.6 times faster and 444.6 times cheaper. TypeSafe's own capabilities team wrote the workflows, and TypeSafe says it cannot prove the price is unsubsidised. Budget from the third-party ratios, and read the vendor's as an upper limit that the vendor set.

2. Multiply by your own volume

Take the per-decision prices from LangChain's run and apply them to an eval suite that grades every change. Suppose 40 changes a day and 20 judged checks per change, which is 800 decisions a day. At $0.05634 per decision the language-model judge costs $45.07 a day, or $1,352.16 in a 30-day month. At $0.00068 per decision Jev costs $0.54 a day, or $16.32 a month. Reruns multiply again: a change that fails twice and passes on the third attempt is graded three times.

Now a guardrail, which is a check that runs on every message a product sends or receives. One million messages a month at Openlayer's per-sample prices is $2,600 through the six-call Ragas pipeline, $460 through one call to GPT-4.1-mini, and $30 through Jev. Openlayer measured Jev's per-request latency at a median of 244 milliseconds and 371 milliseconds at the 95th percentile, meaning 19 requests in 20 finished under that. A user waiting on a guardrail before every reply feels the difference between that and the 2.16 to 2.83 seconds per call LangChain measured for the language models.

Time is the second multiplier, and it is paid in engineer hours rather than dollars. A suite that waits on its grader for two to three seconds per check, 20 checks per change, adds a minute to every run, and the engineer who submitted the change waits for it. On our own suite, with Jev as the grader, the run is ten times faster than it was with the previous language model. We wrote up what changed in our suite. The dollar saving mattered less to us than the minute.

3. When the saving pays for a migration, and when it does not

The migration has a fixed cost, and the saving is per decision, so the answer is the volume.

It pays when decisions arrive in the hundreds a day or more: an eval suite that runs on every change, a guardrail on every message, a routing decision on every request. At 800 decisions a day the saving is $1,335.84 a month. At a million messages a month it is $2,570. Divide the engineer time the migration will take, in dollars, by that monthly saving, and you have the number of months until it has paid for itself.

It does not pay at a few decisions a day. A team that grades 20 decisions a day pays $1.13 a day for the language-model judge, or $33.80 in a 30-day month, at LangChain's per-decision price. Labelling a calibration set, the first hidden cost below, takes more engineer time than a year of that saving buys. Keep the language model and spend the afternoon elsewhere.

It also does not pay, at any volume, for the checks where a probability is the wrong output. A grade that a regulator or an auditor will want explained in words, a design judgment, or a failure nobody wrote a question for all need a language model or a person, and we have written separately about what to do with a grader that gives no reason. And it does not pay for checks that fall on Jev's published weaknesses: counting, dates, numeric comparison. Those go in code, which is cheaper than either model.

4. The four costs the price list leaves out

A calibration set. Before any Jev grade may fail a change, someone on your team labels a set of past judged checks by hand, runs the same checks through Jev, and sets each threshold from the disagreements. This is engineer time, it happens before the first dollar is saved, and it repeats in part every time a rubric changes. Nothing on the price list mentions it, and in our own switch it was the largest cost.

A fallback path. TypeSafe's API documents a 529 error for "temporarily overloaded" and a 429 for the rate limit, and tells you to retry with exponential backoff, meaning growing waits between attempts. The rate limits are 1,200 requests per minute and 250,000 tokens per second. Openlayer reports that in one of its runs the connection service it used hung on a few connections and the 95th-percentile latency for that run rose to a minute. Your suite needs a path for the hour Jev is unavailable: a queue, the previous language-model grader, or a person. Keeping the old grader connected costs something, and it is the price of the suite never stopping.

Version pinning. The current model is jev-1.13.0. The names jev-latest and jev-preview are aliases, which means they point at whichever version is current: both point at 1.13.0 today and will point at the next version when it is released. Name the exact version in your configuration. Every version change means rerunning the labelled set before the new version may fail anything, because a threshold calibrated on one version is a guess on the next.

An early-access vendor. Jev was announced on 15 September 2026 and left its waitlist on 20 September. TypeSafe has not disclosed the architecture. The model reads text only, English is its primary training language, and a request is capped at 64,000 tokens with 32,000 for the state plus the longest question. Openlayer's own tool is an early alpha release, and its direct connection to TypeSafe was tested against a simulated server rather than run live. And the vendor says it cannot prove its price is unsubsidised. A budget built on $0.042 per million tokens should carry a line for the day that number changes.

At eval or guardrail volume these four costs belong in the budget next to the saving, so the person approving it sees both. Our guide to evals with Jev covers the calibration step, and the guide to decision models for product teams covers where else in a product a per-decision price applies. For the wider question of what AI tooling should cost a team, our earlier post on how to budget for AI coding tools still applies.

Credit to the LangChain and Openlayer teams for publishing their costs at list price with the workload attached, and to TypeSafe for saying in public what its own benchmark cannot prove.

Approve the price per month, because that is the one you will pay.

Sources

Introducing System One models and Jev: TypeSafe AI's announcement of 15 September 2026.
Models and API: TypeSafe's price, context limits, rate limits, version aliases and error codes.
Model jaggedness, jev-1.13: TypeSafe's published list of known weaknesses.
Can Jev be a better agent evaluator?: LangChain, 20 September 2026, 500 decisions per judge, $0.34 against $28.17, with the authors' caveats.
jevals: Openlayer's benchmark table, $2.60, $0.46 and $0.03 per 1,000 samples, its latency figures and its alpha status.
TypeSafe AI releases Jev: MarkTechPost, 19 September 2026, on the 444.6 times claim, who wrote the workflows, and the undisclosed architecture.
Jev on System One Models: independent listing recording the waitlist removal on 20 September 2026.

The real price of an LLM judge is per decision, and it multiplies