How to write the state and the questions

Most of the difference between a Jev integration that works and one that disappoints is in the request. The model is fixed. What you put in the state, and how you phrase the questions, is the part you control, and TypeSafe's own documentation is unusually direct about what goes wrong. This page turns that documentation into a set of habits.

What the state can be

The state is the text the model reads. It can be a string, a JSON object, or an array of text values [1]. It is text only: no images, no audio, no files. If your decision depends on a screenshot, something has to describe the screenshot in words first, which is what Browser-use did when it built a browser agent on Jev that reads a structured table of page elements instead of pixels [2].

The size limit is 64k tokens for the whole request, of which the state plus the longest question may use 32k [3]. A token is a piece of a word, so 32k tokens is a long document. TypeSafe's own batching cookbook uses the 53,777-character Wikipedia article on the GDPR as its state [4], and that fits. What does not fit is a whole codebase, a full day of chat logs, or a PDF of a contract, and the right response to that is to select before you send rather than to truncate at the limit.

The three shapes are for your convenience. A string is right for one document. A JSON object is right when the state has named parts the questions will refer to, such as a support ticket with a subject, a body and a customer tier, because the question can then say "the body" and the model knows which part you mean. An array is right for a list of items the questions treat the same way, such as the messages in a conversation.

Keep unrelated content out

TypeSafe's jaggedness page for jev-1.13 states that "Accuracy falls as the state grows with content unrelated to the decision" [5]. This is the single most common mistake we see and the easiest to fix. A team has a large object on hand, so it sends the whole thing, and the model's accuracy on the one field that matters goes down because it has to find that field among forty others.

The rule is to send what the decision needs and nothing else. If the question is whether a diff stays within the files the plan listed, the state is the diff and the plan. The pull request description, the CI log, the commit history and the conversation that produced the plan each add text the model does not need, and each is text that can move its answer.

There is a security reason as well. TypeSafe documents that adversarial content in the state "can move the answer" [5]. Every field you include is a field a hostile user might be able to write into. A support ticket body is user-controlled. A customer tier read from your own database is not. If the decision can be made from the fields you control, make it from those, and treat every user-controlled field as something the model reads with the same literalness it reads your questions. Where Jev is weak covers the mitigation in more depth.

Do the arithmetic in code

Three of TypeSafe's documented weaknesses are about numbers. Jev "does not count reliably", with error growing as the count grows. It cannot reliably judge whether two numeric values are near each other. And it "reads dates as text, not as ordered quantities" [5].

The fix for all three is the same. Compute the number in code and put the result in the state as a plain fact. If the question is whether a customer has contacted support more than three times this month, count the contacts in code and write "contact_count_this_month: 5" into the state. If the question is whether an invoice is overdue, compare the dates in code and write "days_overdue: 12". Then ask the model the judgment that code cannot make, such as whether the tone of the fifth message suggests the customer is about to leave. TypeSafe's design guidance says exactly this: "Keep control flow, deterministic rules, and side effects in code" [6], and the deterministic rules include every comparison of two numbers.

Ask atomic questions

TypeSafe's design guidance names three anti-patterns, and the first is broad questions, with "Is this spam?" as the example [6]. The recommended shape is to break a broad judgment into atomic questions asked in parallel, then combine them in code with weighted arithmetic.

Here is what that looks like for spam. Instead of one question, five Nouls: the message contains a link to a domain not in the allow list; the message asks the recipient to send money or credentials; the message is addressed to no one in particular; the sender's address does not match the display name's organisation; the message repeats the same phrase more than once. Each of those is a fact about the text that the model can read off directly, and each returns its own probability. Code then weights them: a credential request counts for more than a generic greeting. The combined score is yours to tune, and when it is wrong you can see which of the five questions misfired, which you cannot do with a single "Is this spam?" that returned 0.71.

The second anti-pattern is hidden context: relying on what the model knows instead of what is in the state [6]. If your question is "Is this a valid product code?", the model does not have your product catalogue. Put the catalogue, or the relevant part of it, in the state, or check the code in your own database before you ever reach the model. The third anti-pattern is an agent loop where deterministic code would do [6]: if the next step follows from the last answer by a rule you can write, write the rule.

Write each question as one positive clause

The jaggedness page's first entry is the one that catches most people: Jev "answers the question you wrote, not the one you meant. Scoping words, negations, and implied conditions are read at face value" [5]. It adds that double negatives and indirection are answered less reliably.

So each question should be one clause, stated positively, about something present in the state. Some pairs to show the difference.

Write "The response answers the question the user asked." Do not write "The response is not off topic." The second has a negation and an implied standard for what counts as on topic.

Write "The diff changes only files listed under plan.files." Do not write "The diff does not touch anything outside the plan." The second is a negation over an implied scope.

Write "The message asks the recipient to enter a password on a linked page." Do not write "The message is not a normal request." The second asks the model to define normal.

Write "The customer states an intention to cancel the subscription." Do not write "The customer is unhappy." The second is a broad judgment where you wanted a specific fact.

Scoping words deserve their own care. "The response mentions a competitor" and "The response mentions a competitor by name" are different questions, and the model will answer whichever you wrote. If you mean by name, say by name. If you mean by name or by product, say that.

Match the vocabulary of the state

The model reads the question and the state together, and a question that uses different words from the state adds a translation step. If your tickets say "refund" and your question says "reimbursement", you have asked the model to decide that the two mean the same thing before it can answer. Use the state's own words. When the state is a JSON object, refer to its keys by name in the question: "the value of customer.tier is enterprise" reads more precisely than "the customer is an enterprise customer".

The same applies to Choice options and Score level descriptions. Describe each level in the words a reader of the state would use, and make each level clearly more of one thing than the level below it, because a Score assumes order.

A worked request

Take an eval check in a coding pipeline. The specification has an acceptance criterion: "The export endpoint returns a CSV with one row per active user, with the columns id, email and created_at." The agent has produced a diff. The plan listed the files it was allowed to change.

The state is a JSON object with three keys: criterion, which is the sentence above; diff, which is the unified diff; and plan_files, which is the list of file paths. Nothing else. The questions are four Nouls. in_scope: "Every file changed in diff appears in plan_files." adds_columns: "The diff adds the columns id, email and created_at to the CSV output." filters_active: "The diff restricts the exported rows to users whose status is active." has_test: "The diff adds or changes a test that calls the export endpoint." Each is one positive clause about something in the state. Each returns a probability. Code fails the change if in_scope is below its threshold, since a diff outside the plan is never acceptable, and combines the other three with the weights the team chose.

This is the shape Reveneau uses. Every change we make has to pass an eval suite written from the specification, and the checks that used to need a language model judge are now questions to Jev in exactly this form: the criterion, the diff and the plan in the state, one Noul per criterion, deterministic checks kept in code. On our own suite the run is ten times faster than it was with the previous grader, and the request shape is a large part of why. How Reveneau uses Jev has more, evals with Jev is the how-to, and writing specs an agent can verify covers the criterion side.

A checklist before you send

Is every field in the state something the decision needs? If not, remove it. Is any number in the state computed in code rather than left for the model to count or compare? If not, compute it. Is each question one clause, stated positively, with no scoping word you did not mean? If not, rewrite it. Does each question use the state's own vocabulary? If not, align them. Is any user-controlled field in the state something a hostile user could use to steer the answer? If so, decide whether the decision can be made from fields you control instead. Does each question have a name that says what the code will do with the answer? If not, rename it.

Those six checks cover every entry on TypeSafe's jaggedness page except the last, that Jev is not trained to generate text, and that one is covered by never asking it to. The pillar, Jev and System One models, puts this page in context, what Jev returns covers the three question types, and many questions in one call is the next page to read.