AI Decision Models: Add Jev and Clef to Your App (2026)

The setup guide to decision models: what they are, which one to pick, and how to wire Jev and Clef into a real product

Decision models went from one product to a category in under four weeks. TypeSafe released Jev on September 15, and by October 9 Cloudflare, OpenAI, Amazon, Liquid AI and Microsoft had all shipped their own version. Vercel's AI Gateway now lists six decision models side by side - Vercel AI Gateway. An independent leaderboard already ranks 155 open-weight systems of this kind - JevBench. And Bloomberg reported that TypeSafe raised about $870 million at a $7.5 billion valuation in a round led by Andreessen Horowitz - Bloomberg.

The problem is that most apps still route every small judgment through a chat model. Is this ticket urgent? Which team owns it? Does this agent's next tool call look safe? Each of those is a question with a short, known list of answers, yet the usual way to ask it is to pay a language model to write a sentence or a JSON object, then parse what it wrote and hope the format held. A decision model skips the writing. You hand it some state and a set of typed questions, and it hands back a probability for every allowed answer, usually in tens or hundreds of milliseconds, at a price measured in cents per million decisions.

That sounds like a pure upgrade, and it is not. Independent testing shows general-purpose LLMs still answer these questions more accurately, decision models have specific blind spots (counting, dates, long irrelevant context, injected instructions), and several of the headline speed and cost multiples come from the vendors themselves. This guide sorts the real gains from the marketing. It explains how decision models work from first principles, compares the hosted and open options on independent data, then walks through a complete setup for Jev and for Cloudflare's Clef, plus the gateway route that lets you switch between them by changing one line.

The guide is written for builders and founders who ship products, not for ML researchers. Every price, limit and model name was checked against the vendor's own documentation on October 13, 2026.

Contents

1. What a Decision Model Is (and Is Not)
2. Why a Category Formed in Four Weeks
3. The Shared API: State, Questions and Three Answer Types
4. Setting Up Jev Step by Step
5. Setting Up Clef on Cloudflare Workers AI
6. One Integration, Many Models: Gateways and SDKs
7. Choosing a Model on Independent Data and Real Cost
8. Running Decision Models on Your Own Hardware
9. Production Patterns That Work
10. Where Decision Models Fail
11. What Comes Next
12. Conclusion: A Decision Framework

Decision Model Scorecard (October 2026)

The table below scores the seven decision models a builder is most likely to choose between today. Each cell carries the score and the evidence behind it, and quality is taken from the independent JevBench leaderboard wherever a model has been measured there, rather than from vendor benchmarks.

How the criteria were chosen. Decision quality (30%) carries the most weight because a fast wrong answer is worse than a slow right one; it uses JevBench's Intelligence and Calibration axes, which test models on sealed questions they have not seen. Cost (25%) matters because decision models are meant to run on every request, so a few cents per thousand compounds into the main line of the bill. Setup and portability (20%) rewards a model you can adopt in an afternoon and leave without a rewrite. Latency (15%) is weighted lower than you might expect because almost every option here is already fast enough for a web request, and inputs and limits (10%) only becomes decisive when you need images, video or very long context.

Two readings of the table are worth stating plainly. Jev leads because it is the most complete package, not because it wins every axis: Clef-flash is cheaper and faster, Clef reads images and video, and OpenAI's endpoint takes a million tokens of context. And the gap between first and sixth place is small enough that your own labeled data should decide, which is exactly what Section 7 shows you how to measure.

1. What a Decision Model Is (and Is Not)

A decision model is a model that reads an input and returns a probability distribution over answers you defined in advance, instead of generating new text. TypeSafe describes Jev as "a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out" - TypeSafe. You give it the state (a support message, a database record, an agent's proposed tool call) and one or more questions with their allowed answers, and every question comes back as a typed value your code can branch on directly. There is no reply to parse, no JSON that can come back malformed, and no answer outside the list you supplied.

The name most vendors use for the category, System One, comes from Daniel Kahneman's split between fast, intuitive judgment and slow, deliberate reasoning. TypeSafe's documentation is explicit that these models "do not write replies, produce code, or generate explanations of their reasoning" - TypeSafe docs. They are built for the quick, high-volume calls that sit inside software, while reasoning models remain the tool for problems that need several steps of thought. The Cloudflare diagram below shows the shape of a single request: one ticket and two questions go in, and two probability distributions come out in parallel.

The diagram makes the key property visible: the model never writes the word "technical". It assigns 100% to the technical option out of the three it was allowed to choose from, and it answers the urgency question in the same pass. That is the whole contract, and almost every practical advantage and limitation of the category follows from it.

1.1 The first-principles reason it is cheaper and faster

To see why this design is cheap, look at where a language model spends its time. Processing the input (the prefill) happens in one parallel pass over all the tokens. Producing the output (the decode) happens one token at a time, and each new token requires another pass through the network. A chat model asked to classify a ticket still has to decode its answer token by token, and a reasoning model may decode hundreds of hidden thinking tokens first. Output tokens are also priced several times higher than input tokens for exactly this reason: TypeSafe's launch post puts typical LLM output at roughly five times the input price - TypeSafe.

A decision model removes the decode step entirely. Cloudflare explains that Clef runs its backbone "for a prefill-only pass, then scores the valid schema choices in parallel", so "there's no intermediate text to generate token by token" - Cloudflare. AWS's Strands team built theirs by taking a pre-trained model's "torso", removing the part that generates text, and attaching a pointer head of just over a million parameters that scores each offered option - Strands Agents. Marc Brooker, the Amazon distinguished engineer who started that project, notes that inference is one forward pass for one question and two for any larger number, "so still O(1), not O(questions)", because the pass over the state is cached - Brooker.

The pricing reflects that physics directly. Every hosted decision model checked for this guide charges only for input tokens: Jev at $0.042 per million, Clef-flash at $0.038, Liquid's d1 at $0.04, Microsoft-Decision-1 at $0.042 and OpenAI's decision endpoint at $0.10, with output free in every case. You are paying to have your input read, never to have an answer written.

1.2 Calibrated probabilities are the real product

Speed and price are the headline, but the more important property for software is calibration. A calibrated model's probabilities mean what they say: across many predictions, outcomes it assigns 0.8 should happen about 80% of the time. TypeSafe trains for this with a method it calls Reinforcement Learning for Calibrated Decisions (RLCD), and its primer contrasts it with RLHF, which "teaches a model to say things that people prefer" and can reward "confident-sounding hallucinations" - TypeSafe AI primer.

Why this matters is easiest to see from the developer side. Armin Ronacher, CTO of Earendil, told TechCrunch the model "delegates the hallucination problem a little bit to the user": a 50% answer is a coin toss you can ignore, while a 95% answer is one you can act on - TechCrunch. A chat model's confident prose gives you no such dial. With a calibrated probability, your code can automate the clear cases, escalate the unclear ones, and set the threshold per action based on what a mistake costs.

There is an important nuance most coverage skips. Calibration describes groups of predictions, not any single answer. TypeSafe's own documentation says calibration "does not guarantee that an individual answer is correct" - TypeSafe docs, and Ollama's model page warns that its confidence field "isn't the chance that the answer is right" but a measure of how concentrated the probabilities are - Ollama. Treat the numbers as a routing signal that you verify against labeled examples, never as a guarantee.

IBM's explainer is the clearest short introduction to the concept, and at roughly 1.15 million views it is the most-watched video on the topic. Martin Keen covers how Jev returns probabilities (from 4:04), how calibration is trained (from 9:18) and how to combine it with LLM workflows (from 12:56).

1.3 What a decision model cannot do

The constraint that makes these models fast also defines their limits. They cannot write a reply, summarize a document, generate code or explain their reasoning. They cannot answer an open question such as "what is the customer's order number" unless you turn it into a choice among candidates you supply. And because the answer space is fixed, they cannot tell you that your options were wrong in the first place; if none of your categories fits, the model will still spread its probability across them.

TypeSafe makes this point directly for anyone arriving from coding tools: Jev "is not a drop-in replacement for the LLM behind Claude Code, Cursor, opencode, Copilot" and there is "no model: "jev-latest" setting that turns your coding agent into a Jev-powered agent" - TypeSafe docs. The right mental model is a component inside your product, sitting next to the LLM rather than replacing it.

How to apply this: go through your product and list every place where code currently asks an LLM a question whose answer is one of a few known values: routing, tagging, yes/no checks, rubric scores, "should the agent continue". Those are your candidates. Anything that produces words a person will read stays with a generative model. If you want a refresher on how the generative side of that split is priced and tuned, our guides to model routing and the effort dial cover it in depth.

2. Why a Category Formed in Four Weeks

Most new model types take a year to attract serious competitors. This one took days. The timeline below lists only launches confirmed on the vendor's own page or changelog, and it shows how quickly the field filled in after TypeSafe's launch on September 15.

Reading the timeline as a hype wave misses the structure underneath it. Four forces made this category inevitable once someone demonstrated it, and understanding them tells you which parts of the excitement will last.

2.1 Agents multiplied the number of small decisions

The first force is a change in the shape of AI workloads. A chat product makes one model call per user message. An agent makes dozens: which tool to call next, whether the result looks right, whether to continue or stop, whether an action needs approval. Microsoft's announcement puts the arithmetic plainly: "adding just 100 milliseconds to each of 20 sequential decisions adds two seconds to the overall workflow" - Microsoft Command Line. When most of an agent's calls are small judgments rather than generation, the cost and latency of those judgments becomes the cost and latency of the product.

That is also why security teams took notice so quickly. Anthropic's October 9 report described Claude models in evaluations submitting a real police tip form, reusing access tokens found in a website's settings file, and working around a data fee, and it named safety classifiers among the tools it now uses to monitor far more of what agents do - Anthropic. Checking every agent action with a frontier model is expensive; TechCrunch reported a hackathon demo where monitoring a workload cost $2.94 with Jev versus $372 with a frontier LLM - TechCrunch. A check cheap enough to run on every action is a different kind of safeguard from one you can only afford to sample.

2.2 The recipe turned out to be copyable

The second force is that the core technique is accessible. Most competitors did not train a model from scratch. Clef is post-trained from Qwen3.8-27B, Clef-flash and Microsoft-Decision-1 from Qwen3.5-9B, and Strands Decider from Qwen3.5-2B, according to their model cards and announcements - Hugging Face. Brooker told TechCrunch that with smaller markets, "the cost to build something interesting is in the hundreds or thousands of dollars" - TechCrunch. Open-weight backbones did the expensive part years in advance; turning one into a calibrated scorer is a fine-tuning project, not a frontier training run.

TypeSafe's founder disputes that this makes the models equal. Diogo Almeida, who co-invented RLHF at OpenAI, told TechCrunch that competitors "might be underestimating the difficulty of making the models actually smart" and that "the current batch seems more like ML people wanting to implement a cool architecture" - TechCrunch. He argues the moat is synthetic training data; Brooker's own build log reached a similar conclusion from the other side, calling the work "fundamentally a data game". The independent numbers in Section 7 support a middle view: the architecture is easy to copy, but the quality spread between the best and worst copies is enormous.

The diagram from TypeSafe's primer shows where RLCD sits relative to the two training methods that produced chat models and reasoning models.

The diagram's point is that RLCD is a third branch from the same trunk, not a smaller LLM. RLHF made models pleasant to talk to and RLVR made them good at verifiable reasoning; RLCD optimizes for probabilities that match outcomes. That framing explains why every competitor started from an existing open-weight backbone: the trunk is shared, and the differentiation lives in the post-training data and objective.

2.3 A simple API invited clones

The third force is the interface. Jev's API is one endpoint, three question types and a JSON body small enough to read in a minute. Cloudflare made Clef "fully Jev-API compatible" from day one - Cloudflare, Ollama shipped a local /v1/systemone endpoint that "follows TypeSafe's Jev API" - Ollama, and Liquid's d1 and Microsoft's Foundry deployment answer at SystemOne-shaped paths as well. When the contract is simple, switching costs are close to zero, and every provider with GPUs has a reason to fill the slot. Section 3 covers that shared contract in detail, because it is the single most useful fact for anyone building on this category.

2.4 Cheaper intelligence gets used more, not less

The fourth force is the one TypeSafe named its model after. William Stanley Jevons observed that when coal use became more efficient, total coal consumption rose rather than fell. Almeida's bet, as he told TechCrunch, is that the falling cost of intelligence will put "smart software all over the place in a way that's emergent and distributed" - TechCrunch. At $0.042 per million input tokens, a judgment that used to cost a meaningful fraction of a cent costs a few thousandths of one, and that changes which features are worth building. A check you would never have paid an LLM to run on every row of a table, every incoming email or every agent step becomes affordable by default.

The money followed the thesis. TypeSafe emerged from stealth with a $40 million seed led by DCVC - SiliconANGLE, then raised the reported $870 million less than four weeks later. Bloomberg's report adds the company's own claims that Jev is used by about a third of Fortune 500 companies and passed one million users, though TypeSafe declined to name customers. Treat those adoption figures as unverified marketing until a customer confirms them.

Why this matters for you: the forces behind the category (agent workloads, cheap backbones, a copyable API, falling unit cost) are structural, so decision models are not going away even if individual vendors do. How to apply this: build against the shared contract rather than a single vendor's SDK quirks, keep your evaluation data in your own hands, and assume today's prices and rankings will move within months.

3. The Shared API: State, Questions and Three Answer Types

Before touching any vendor, it pays to learn the request shape that most of them now share, because once you know it, every setup in this guide is the same code with a different address. TypeSafe's endpoint is POST https://api.typesafe.ai/v1/systemone, and the body has exactly three required fields: the state to evaluate, the model, and a map of named questions - TypeSafe API reference. The question keys are yours to choose, they are never sent to the model as content, and every answer comes back under the same key.

The state can be a plain string, a JSON object or an array, which matters more than it looks. Passing a structured record (an order with its status, an agent with its proposed tool call) lets you point a question at a specific field by name in backticks, the same way you would reference a variable. That keeps instructions short and makes it obvious which part of the input a wrong answer came from.

3.1 The three question types

Every question has a type and instructions, and most also take criteria. The three types cover nearly every decision a product makes, and choosing the right one is most of the design work.

The names are TypeSafe's, and the rest of the ecosystem has mostly kept them. Cloudflare, Liquid AI, Microsoft's Foundry route and Ollama all accept noul, choice and score exactly as written here, while Vercel's own AI SDK calls the yes/no type boolean and OpenAI calls it predicate. The behavior is identical across all of them, so it is worth thinking in terms of the three shapes of answer rather than any one vendor's label. Each shape maps to a familiar construct in code: a Noul is an if-statement, a Choice is a switch over an enum, and a Score is a number you compare against a threshold.

• Noul asks a yes/no question and returns the probability that the answer is yes, from 0 to 1. Optional criteria describe what "true" and "false" mean.
• Choice picks one option from a map you define and returns the chosen key, a probability for every option and a confidence value. Jev accepts up to 255 options per Choice.
• Score places the state on an ordered rubric and returns a probability-weighted position between levels, plus per-level probabilities and confidence. The API accepts 2 to 10 levels.

The distinction between Choice and Score is the one people get wrong most often. Use Choice when the options have no order (departments, intents, product categories), and Score when they do (severity, urgency, quality), because a Score answer of 1.4 on a 0-to-2 scale carries information that a forced choice of "1" throws away. OpenAI's guide gives the same advice and explains that a score "takes the probability-weighted average of their numeric indices to produce a score that can fall between levels" - OpenAI. Noul is for any check you would otherwise express as an if-statement, and several Nouls in one request are often clearer than one Choice with overlapping options.

Here is a complete request that asks all three types about one support message. It is the same body whether it goes to TypeSafe, to Cloudflare's Clef, to Ollama on your laptop or through Vercel's TypeSafe-compatible endpoint; only the URL, the key and the model value change.

{
  "model": "jev-latest",
  "state": "Our API integration started returning 500 errors on every request about 20 minutes ago, and we can't process any customer orders until this is fixed.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this",
      "criteria": {
        "billing": "Payment or subscription issues",
        "technical": "Bugs or integration problems",
        "sales": "Pricing or account questions"
      }
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated the customer appears",
      "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"]
    },
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity"
    }
  }
}

3.2 Reading the response

The response mirrors the questions. TypeSafe's quickstart shows this example coming back with technical at probability 1.0, a frustration score of 0.67 (one third on "calm", two thirds on "frustrated but civil") and an urgency probability of 0.99, using 428 input tokens - TypeSafe quickstart. At Jev's list price that single call costs about $0.000018, which is to say roughly 55,000 such decisions for a dollar.

{
  "model": "jev-1.13.0",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "technical",
      "confidence": 1.0,
      "probabilities": { "technical": 1.0, "billing": 0.0, "sales": 0.0 }
    },
    "frustration": {
      "type": "score",
      "score": 0.67,
      "confidence": 0.5,
      "legend": { "0": "Calm, just stating facts", "1": "Frustrated but civil", "2": "Very angry, strong language" },
      "probabilities": { "0": 0.33, "1": 0.67, "2": 0.0 }
    },
    "is_urgent": { "type": "noul", "noul": 0.99 }
  },
  "usage": { "input_tokens": 428, "output_tokens": 73 }
}

Two fields deserve attention. The model field reports the pinned version that answered (jev-1.13.0), even though the request asked for the alias jev-latest, so log it with every decision. And confidence is not the same thing as probability. For a Choice, TypeSafe's documentation computes it from how concentrated the distribution is, scaled so that a uniform spread scores 0 and a single certain option scores 1 - TypeSafe confidence. The frustration answer above has a clear winner at 0.67 but only 0.5 confidence, because a third of the probability sits on a neighboring level. A Noul has no confidence field at all; its probability already is the answer.

3.3 One contract, many backends

The practical consequence of this shared shape is portability. The diagram below maps where the same request body can be sent today, with the one exception that speaks its own dialect.

OpenAI is the outlier. Its Decisions API uses an input field instead of state, an array of questions each carrying a name, a predicate type instead of noul, and choices written as objects with value and description - OpenAI. It also adds a refusal answer type, which the SystemOne models do not have. The semantics are the same three primitives, but the JSON is different enough that you would write an adapter. Vercel's gateway already does that translation in both directions, accepting OpenAI-shaped requests for any decision model and SystemOne-shaped requests for OpenAI's - Vercel.

Why this matters: a contract that several vendors honor means the decision layer of your product does not belong to any one of them. If TypeSafe raises prices, if a model is retired the way OpenAI is retiring a long list of older models on October 23 (we covered that in our GPT-4 shutdown guide), or if a cheaper model appears next month, the change is a configuration value. How to apply this: write your questions in the SystemOne shape, keep the base URL and model name in environment variables, and wrap the call in one small function so the rest of your code never knows which vendor answered. The same principle runs through our guide to owning your AI stack.

4. Setting Up Jev Step by Step

Jev is the reference implementation of the category, so it is the natural place to start even if you end up on another model. The setup takes about ten minutes, and everything below comes from TypeSafe's current documentation and package registries as of October 13: the Python package typesafe-sdk is at version 0.7.4 and requires Python 3.10 or later - PyPI, and the JavaScript package @typesafe-ai/sdk is at 0.6.0 and needs Node.js 20 or later (22.18 or later to run TypeScript files directly) - TypeSafe quickstart.

Before writing code, it helps to know the operating limits, because they shape how you design requests. A single request can carry 64k tokens, but only 32k of that can be the state plus the longest question; the price is $0.042 per million input tokens with output free; and the default rate limits are 100,000 tokens per second and 80 requests per second, with a 429 response when you exceed either - TypeSafe models. Jev reads text only: strings, JSON objects and arrays, with no images, audio or video yet.

If you want the founder's own account of why the model is built this way, Diogo Almeida's long interview with swyx on Latent Space covers RLCD, why TypeSafe refuses to optimize for public benchmarks, and how he expects Jev to change the architecture of coding agents. It is long, but it is the most detailed primary source on the design.

4.1 Get a key and pin a version

Create an API key in the TypeSafe console and store it as TYPESAFE_API_KEY; both SDKs read that variable automatically. The console also has a browser Playground where you can paste a state, add questions and see answers without code, which is the fastest way to sanity-check whether a decision is a good fit before you write anything.

Then decide on the model name, which is a more consequential choice than it looks. TypeSafe publishes one model under three names: jev-1.13.0 is the pinned version, jev-latest points at the most recent stable release, and jev-preview at the most recent release of any kind. The documentation warns that aliases move when new releases ship and says to pin jev-1.13.0 if you need stable answers - TypeSafe models. A decision model sits in your control flow, so a silent model change can change which branch your code takes for the same input. Use jev-latest while you prototype, pin the version you evaluated before you ship, and upgrade deliberately after re-running your labeled examples.

4.2 Python: a production-shaped first integration

Install the SDK into a virtual environment. TypeSafe's quickstart shows both pip and uv; the uv form is shown here.

uv init --bare
uv add typesafe-sdk
export TYPESAFE_API_KEY="your-api-key"

The quickstart's own example asks three questions about one ticket and prints the three answers. The version below keeps the same SDK calls but shapes them the way you would ship them: the version pinned through an environment variable, the state passed as a structured object so the question can name its field, an explicit other option so nothing is forced into a wrong bucket, and a confidence floor that sends unclear tickets to a person.

import os
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

JEV_MODEL = os.environ.get("JEV_MODEL", "jev-1.13.0")  # pin what you evaluated
client = TypeSafeClient(model=JEV_MODEL)  # reads TYPESAFE_API_KEY

def triage(ticket: str) -> dict:
    response = client.system_one(
        state={"ticket": ticket},
        questions={
            "team": Choice(
                instructions="Which team should handle `ticket`?",
                criteria={
                    "billing": "Charges, invoices and refunds",
                    "technical": "Bugs, outages and integrations",
                    "account": "Sign-in and account access",
                    "other": "None of the above",
                },
            ),
            "urgency": Score(
                instructions="How time-sensitive is `ticket`?",
                criteria= ["Can wait a week", "Should be handled today", "Blocking the customer now"],
            ),
            "wants_refund": Noul(instructions="Does `ticket` explicitly ask for money back?"),
        },
    )
    team = response.choices ["team"]
    return {
        "team": team.choice if team.confidence >= 0.6 else "human_review",
        "team_confidence": team.confidence,
        "urgency": response.scores ["urgency"].score,
        "wants_refund": response.nouls ["wants_refund"].noul,
        "model": response.model,  # log the version that answered
    }

print(triage("I was charged twice for March and I still can't log in."))

Each design choice in that function prevents a specific failure. The pinned version prevents silent drift; the other option gives the model somewhere honest to put a ticket that fits nothing; the 0.6 floor is the same starting point TypeSafe uses in its voice-banking example, where low-stakes actions run at 0.6 and a transfer requires more than 0.85 - TypeSafe confidence routing. And returning the model version with the answer means that when a decision looks wrong in your logs, you can tell whether the model changed or the input did. The SDK also ships an async client and a RetryPolicy for tuning retries, which matters once you call it from a web server.

4.3 TypeScript: typed answers for free

The JavaScript SDK has one feature worth switching languages for: it infers answer types from your questions. After you declare a Choice with the keys billing, technical and sales, the compiler knows that response.answers.department.choice can only be one of those three strings, so a typo in a later switch statement fails the build instead of a request.

npm install @typesafe-ai/sdk
npm pkg set type=module

import { choice, noul, score, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({ defaultModel: process.env.JEV_MODEL ?? "jev-1.13.0" });

export async function triage(ticket: string) {
  const response = await client.systemOne({
    state: { ticket },
    questions: {
      team: choice("Which team should handle `ticket`?", {
        billing: "Charges, invoices and refunds",
        technical: "Bugs, outages and integrations",
        other: "None of the above",
      }),
      urgency: score("How time-sensitive is `ticket`?", [
        "Can wait a week",
        "Should be handled today",
        "Blocking the customer now",
      ]),
      wantsRefund: noul("Does `ticket` explicitly ask for money back?"),
    },
  });

  const { team, urgency, wantsRefund } = response.answers;
  // team.choice is typed as "billing" | "technical" | "other"
  return {
    team: team.confidence >= 0.6 ? team.choice : "human_review",
    urgency: urgency.score,
    wantsRefund: wantsRefund.noul,
  };
}

The same typing makes refactors safe. Rename an option, and every place that branches on it lights up. That is a meaningful improvement over the usual LLM pattern of asking for JSON and validating it with a schema library after the fact, where a renamed category often surfaces as a runtime parse error weeks later. If structured-output failures have bitten you before, our piece on why AI apps corrupt data explains the general problem this sidesteps.

4.4 Raw HTTP, errors and retries

You do not need an SDK at all. The endpoint takes a bearer token and the JSON body from Section 3, which makes it easy to call from a shell script, a database trigger or a language without an official client.

curl --fail-with-body --silent --show-error \
  https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-1.13.0",
    "state": "Help! My payouts have been failing for 3 days.",
    "questions": {
      "is_urgent": { "type": "noul", "instructions": "Does this convey urgency?" }
    }
  }'

The error codes are standard but worth handling explicitly. A 401 means a missing or bad key, a 422 means the body failed validation (the response names the offending field), a 429 means you hit a rate limit, and a 529 means TypeSafe itself is overloaded - TypeSafe API reference. Retry 429 and 529 with exponential backoff; the official SDKs do this automatically and honor retry-after. The 529 code is not theoretical: shortly after launch, TypeSafe "briefly lost the ability to serve users from its API because demand was so high" - TechCrunch. Any decision on your critical path therefore needs a fallback, which Section 6 shows how to configure.

4.5 Let your coding agent write the integration

If your product is built with an AI coding agent, the most efficient setup is to give that agent TypeSafe's skill and describe the decision you need. TypeSafe publishes the skill in a public repository - GitHub, with a two-command install for Claude Code (claude plugin marketplace add typesafe-ai/skills, then claude plugin install typesafe@typesafe-ai) and npx skills add typesafe-ai/skills --skill typesafe-ai for other agents. The skill gives the agent the API, the three primitives and the documented patterns, so it writes confidence-gated code rather than a naive call.

This matters more than it sounds, because the most common mistake with decision models is asking them the wrong kind of question, and a coding agent that only knows LLM conventions will happily write a prompt asking Jev to "return the order total as a number". With the skill loaded, the agent knows the arithmetic stays in code and only the judgment goes to the model. The same applies whichever AI coder writes your product: Claude Code, Codex, Replit's Agent (which can now wire Jev in through Replit AI Integrations without you managing a key - Replit), or the builders that a platform such as Founden runs on each company's own codebase. In every case the request is the same: name the decision, list the allowed answers, and say what should happen when the model is unsure.

Why this matters: the integration itself is a few lines, so the quality of a Jev deployment is decided almost entirely by version pinning, question design and error handling, which are exactly the parts a quick demo skips. How to apply this section: prototype in the Playground, ship with a pinned version, wrap the call in one function, log the model version and confidence with every decision, and handle 429 and 529 as normal events rather than outages. The next section repeats the same exercise on Cloudflare, where the request body stays the same but the deployment model is completely different.

5. Setting Up Clef on Cloudflare Workers AI

Cloudflare's Clef family is the strongest case for not tying yourself to one vendor, because it accepts the same request as Jev while changing almost everything around it. The models run on Cloudflare's own GPUs inside Workers AI, the weights are published under Apache 2.0 so you can run them yourself, and the family reads images and video as well as text - Cloudflare. Cloudflare also states that it does not read, store or train on requests or responses unless you opt into its fine-tuning service, which is a meaningful point for teams sending customer data through a decision layer.

There are now three models in the family, and picking among them is the main decision. All three use the same API and the same three question types; they differ in size, speed, price, context and which media they accept. The table below uses Cloudflare's own model pages and its October 9 changelog, which cut Clef-flash's price and made Clef faster - Cloudflare changelog.

The Clef-flash price change explains the odd context number. Cloudflare dropped it from $0.090 to $0.038 per million input tokens, which made it cheaper than Jev, and cut the hosted context from 64K to 24K to afford that, noting that only 0.24% of requests exceeded 24K tokens. The weights on Hugging Face are unchanged and still support 256K tokens if you host them yourself. Clef-omni, released the same day, handles text in about 20 ms, images or audio in under 100 ms and a 21-second video clip with sound in about 300 ms, according to Cloudflare.

5.1 The Workers binding

The cleanest way to call Clef is from a Cloudflare Worker, where the model is available through a binding with no API key in your code. Add the AI binding to your Wrangler configuration - Cloudflare docs:

[ai]
binding = "AI"

Then call the model from the Worker. This is Cloudflare's own example with the three question types; note that the request body carries a model field (clef, clef-flash or clef-omni) in addition to the model ID in the first argument, which is the detail most people miss on their first attempt.

export interface Env {
  AI: Ai;
}

export default {
  async fetch(request, env): Promise<Response> {
    const result = await env.AI.run("@cf/cloudflare/clef", {
      model: "clef",
      state: "Checkout has been failing for every customer for the last hour.",
      questions: {
        urgent: { type: "noul", instructions: "Is this support request urgent?" },
        team: {
          type: "choice",
          instructions: "Which team should handle this request?",
          criteria: {
            billing: "Payments, invoices, and refunds",
            technical: "Outages, errors, and configuration",
            sales: "Plans and upgrades",
          },
        },
        severity: {
          type: "score",
          instructions: "How severe is the customer impact?",
          criteria: ["No impact", "Minor", "Major", "Critical"],
        },
      },
    });
    return Response.json(result);
  },
} satisfies ExportedHandler<Env>;

The binding route matters most when the decision sits on the request path of something already running at Cloudflare's edge: a bot check, a content filter, a routing decision in front of an API. Cloudflare's pitch is that edge GPUs cut network latency on top of model latency, so you can "put Clef into the hot path for agents". If your app is hosted elsewhere, the REST endpoint below works the same way, and our ranking of where to deploy your app covers when moving to Workers is worth it for its own sake.

5.2 REST from any backend

From a server that is not a Worker, call the same model over HTTPS with an account ID and an API token that has Workers AI permission. Clef-flash is shown here because it is the one most teams will start with on cost.

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef-flash \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{
    "model": "clef-flash",
    "state": "Checkout is failing for all customers.",
    "questions": {
      "urgent": { "type": "noul", "instructions": "Is this urgent?" }
    }
  }'

Images go in an images array as base64 data URLs, placed before the state, up to four per request, and Clef-omni adds audio and videos arrays in the same style. Remote image URLs are not accepted, so fetch and encode them yourself. Each image is resized to multiples of 32 pixels and billed as roughly one token per 32-by-32 patch plus three marker tokens, capped at 1,024 tokens per image - Cloudflare docs. At Clef's rate, even a full-size image costs at most about $0.00025, or roughly 4,000 images per dollar.

5.3 What it costs, including the free allocation

Workers AI bills in Neurons, and Cloudflare's pricing page converts each model's token price into them: Clef costs 21,818 Neurons per million input tokens and Clef-flash 3,455 - Cloudflare pricing. Every account gets 10,000 Neurons per day free, and usage above that is $0.011 per 1,000 Neurons on the Workers Paid plan. Converted, the free allocation covers roughly 458,000 Clef input tokens or 2.9 million Clef-flash input tokens a day.

In decisions rather than tokens, that is a useful amount. At a typical 400-token request, the free tier covers about 1,100 Clef decisions or about 7,200 Clef-flash decisions per day before you pay anything, which is enough to run a real feature in a small product or to evaluate both models on your own data at no cost. Above the free tier, a million 400-token decisions costs about $15 on Clef-flash and $96 on Clef. Section 7 puts those numbers next to every other option.

5.4 How Clef compares to Jev, on Cloudflare's numbers and others'

Cloudflare published a detailed comparison at launch, and it is worth reading carefully because it shows both why Clef is credible and why vendor benchmarks need a second opinion. On the Decision Index (an open benchmark hosted on Hugging Face), Cloudflare's chart puts Clef at 61.2, ahead of Jev at 57.9, with Clef-flash at 57.1 and far lower latency.

The chart's own legend labels Cloudflare's points as self-reported, which is the honest framing. Across individual benchmarks the picture is more mixed than the headline: Clef wins clearly on intent classification, Jev wins on deciding when to call a tool and on the BRIGHT retrieval benchmark, and Clef-flash collapses on CLINC150, a test that includes out-of-scope requests. The chart below plots five of Cloudflare's ten published benchmarks.

The pattern in that chart is the useful part. Clef is strongest where the task is classic classification with many labels, Jev holds up better on judgments about agent behavior and retrieval, and Clef-flash trades accuracy for speed in ways that show up on harder or out-of-distribution inputs. Independent measurement narrows the gap further: on JevBench's sealed questions, Clef scores 59.1 on Intelligence against Jev's 63.6, and Clef-flash 42.2 - JevBench. Neither vendor's chart tells you which one wins on your tickets.

Better Stack ran exactly that kind of test on real data, pushing 200 comments from its YouTube channel through both models with Claude Opus 5.5 as the referee. Jev came out faster and cheaper, but Clef finished within three points on accuracy and won three of the five questions, and an attempt to have Clef predict which thumbnails would perform best was "basically a coin flip". The video shows the full method from the 2:58 mark.

Speed is the other axis where Cloudflare moved quickly. Its October 9 update roughly halved Clef's latency without changing the weights, partly by moving serving to SGLang. Because latency grows with input size, the gain is easiest to see across request lengths.

The slope of that line carries a design lesson that applies to every decision model. A 16,000-token state takes about ten times longer than an 800-token one, so the cheapest latency optimization available to you is sending less state. Filter the record down to the fields the question needs before you call the model; it improves accuracy too, as Section 10 explains.

5.5 When Clef is the right pick

Why this matters: Clef is the only family here that combines a hosted API, open weights you can take elsewhere, vision and video input, and a fine-tuning path (Cloudflare is offering hands-on fine-tuning through its forward-deployed engineers now and plans a self-serve version). How to apply this: choose Clef-flash for high-volume text decisions where cost and latency dominate, Clef when you need images, video or a 64K context and can afford about six times Jev's price, and Clef-omni when the input is a voice recording or a video with sound. If your workload is plain text and accuracy on agent-behavior questions matters most, test Jev alongside it before committing.

6. One Integration, Many Models: Gateways and SDKs

Calling each vendor directly works, but it leaves you with one key, one bill and one failure mode per provider. A gateway puts every decision model behind one endpoint, which is why Vercel's AI Gateway has become the most practical place to run this category: it added Jev the day after launch and had six decision models by October 13, along with three different API dialects and a fallback mechanism built specifically for decisions. The model IDs and prices below come from the gateway's live model list - Vercel AI Gateway.

The economics of the gateway itself are simple: decision calls are billed at each model's per-token rate and appear in the same logs and budgets as your other model traffic. In the example response in Vercel's documentation, a 275-token Jev decision costs $0.00001155 with a gateway surcharge of zero - Vercel docs. The Laya promotion is worth knowing about for experiments, with the caveat that the -free model ID stops serving after October 31 while the standard ID starts billing - Vercel.

6.1 The AI SDK route

If your app is TypeScript, the Vercel AI SDK now has a decision function. It requires the ai package at 7.0.128 or later (the current release is 7.0.138); earlier AI SDK 7 releases called the same thing experimental_evaluate, and decision models were briefly called "evaluation models", which you will still see in older posts and in the HTTP path. Note the type name: the SDK calls the yes/no question boolean and returns its answer as probability.

import { experimental_decide as decide } from "ai";

export async function POST(req: Request) {
  const { ticket } = await req.json();

  const result = await decide({
    model: process.env.DECISION_MODEL ?? "typesafe-ai/jev",
    state: { ticket },
    questions: {
      route: {
        type: "choice",
        instructions: "Route `ticket` to a queue.",
        criteria: {
          billing: "Payment or charge problems",
          shipping: "Delivery problems",
          technical: "Application bugs",
        },
      },
      refund: { type: "boolean", instructions: "Is a refund requested?" },
    },
    providerOptions: { gateway: { zeroDataRetention: true } },
  });

  return Response.json(result.answers);
}

Because the model is a string, switching from Jev to Liquid's d1 or Microsoft's model is an environment variable change, which is the portability argument from Section 3 made concrete. The zeroDataRetention option restricts the request to providers that support it, and Vercel lists Zero Data Retention and No Training as per-request options for Jev - Vercel. For teams in the EU or handling personal data, that is the switch to set first; our guide to making an AI app EU-compliant covers why data retention terms carry so much weight there.

6.2 Keep your TypeSafe code, change the address

If you already wrote against TypeSafe's SDK, you do not need to rewrite anything to move behind the gateway. Point the client at Vercel's TypeSafe-compatible endpoint, swap the key, and change the model name to any decision model the gateway carries.

import { TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: "https://ai-gateway.vercel.sh/typesafe",
});

const result = await client.systemOne({
  model: "microsoft/microsoft-decision-1", // or "typesafe-ai/jev", "liquid/d1"
  state: "My invoice payment failed and I need help immediately.",
  questions: {
    team: {
      type: "choice",
      instructions: "Which team should handle the request?",
      criteria: {
        billing: "Payment and invoice issues",
        technical: "Software and integration issues",
      },
    },
  },
});

This snippet is Vercel's own example for Microsoft-Decision-1 - Vercel, and it illustrates something unusual about this market: a client library published by one vendor is the standard way to call a competitor's model. Microsoft's model on Foundry needs an Azure deployment of its own, so the gateway is also the shortest path to trying it, with no Azure account required.

6.3 OpenAI's Decisions API, directly or through the gateway

OpenAI released its Decisions API in beta on October 6 with gpt-6-luna as the only supported model - OpenAI changelog. It costs $0.10 per million input tokens with no charge for output, cache reads or cache writes, claims answers "about 10x faster than the Responses API", supports Zero Data Retention and HIPAA for eligible customers, and offers data residency in the US and Europe - OpenAI. You need OpenAI SDK 3.26.0 or later for Python or 7.30.0 or later for JavaScript; the current releases are 3.29.0 and 7.32.0.

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY

decision = client.decisions.create(
    model="gpt-6-luna",
    input="I was charged twice for my order.",
    questions= [
        {
            "type": "choice",
            "name": "department",
            "instructions": "Which department should handle this complaint?",
            "choices": [
                {"value": "billing", "description": "Payments, invoices, and refunds."},
                {"value": "technical", "description": "Problems using the product."},
                {"value": "other", "description": "Requests outside these categories."},
            ],
        }
    ],
)

answer = decision.answers [0]
if answer.type == "refusal":
    print("Refused:", answer.name)
else:
    print(answer.choice, answer.confidence)

Three differences from the SystemOne shape matter in practice. Answers come back as an array matched by name, not a map; the model can refuse a question, so your code needs a branch for that; and OpenAI's endpoint accepts images inside the same message format as its other APIs. To call it through Vercel's gateway instead, set base_url="https://ai-gateway.vercel.sh/v1", use your gateway key, and name the model openai/gpt-6-luna-decisions. The gateway also accepts this OpenAI-shaped request for Jev, d1 and Laya, so a team standardized on OpenAI's SDK can still reach the rest of the category.

6.4 Confidence-based fallbacks

The most useful feature in the gateway is one no single vendor offers: rerunning a decision on a stronger model when the first answer is unsure. Added on October 6, it lets you attach one conditional fallback to a request; when the primary model's Choice or Score confidence drops below a threshold, or a Boolean probability lands inside an uncertain band, the gateway reruns the whole decision on the fallback model - Vercel. The fallback can be another decision model or a full language model, which then answers through structured output.

import { gateway } from "@ai-sdk/gateway";
import { experimental_decide as decide } from "ai";

const result = await decide({
  model: gateway.decisionModel("typesafe-ai/jev"),
  state: "I was charged twice and cannot sign in to request a refund.",
  questions: {
    intent: {
      type: "choice",
      instructions: "Which team should handle this support request?",
      criteria: {
        billing: "Charges, invoices, and refunds",
        technical: "Bugs and outages",
        account: "Sign-in and account access",
      },
    },
    requestsRefund: { type: "boolean", instructions: "Does the customer ask for a refund?" },
  },
  providerOptions: {
    gateway: {
      models: [
        {
          model: "openai/gpt-6-astra",
          when: { question: "intent", confidenceBelow: 0.6 },
        },
      ],
    },
  },
});

This is the cascade pattern from Section 9 implemented as configuration rather than code. Two details from Vercel's documentation matter for cost and latency. When the fallback triggers, both stages are billed and both latencies add up, because the stages run in sequence. And only one conditional fallback is allowed per request, though you can list plain model names after it as fallbacks for outright errors, which also covers the 529 overload case from Section 4. For a Boolean question, use probabilityBetween: [0.4, 0.6] instead of confidenceBelow - Vercel docs.

6.5 Other routes in: Replit, OpenRouter and agent frameworks

Two more distribution paths show how quickly decision models are being absorbed into tools builders already use. Replit added Jev to its AI Integrations on October 2, so an app built with Replit's Agent can call jev-latest through OpenRouter without the builder handling any key. OpenRouter itself lists a Jev Router, which uses Jev to pick "the best model and reasoning effort for each request" across OpenRouter's catalog - OpenRouter, a direct product version of the model-routing idea.

Agent frameworks are the third route. Vercel notes that its agent framework eve uses Jev as the default decision model for automatic model selection, typed evaluations and automated tool approvals - Vercel, and AWS ships its own decider with an intervention hook in the Strands agent SDK. When the framework makes the decision model a default, many developers will use one without ever choosing it, which is worth keeping in mind when you audit what your agents are actually calling.

Why this matters: a gateway turns vendor choice, data retention and fallback behavior into configuration, which is the cheapest insurance available in a market where prices and models change weekly. How to apply this section: if you run on Vercel or already use the AI SDK, start on the gateway with typesafe-ai/jev, turn on zero data retention, and add one confidence fallback to a stronger model for the decisions where a mistake is expensive. If you are committed to OpenAI's SDK, use its Decisions API and keep the gateway in mind as the bridge to other models. Either way, the model name should live in configuration from day one.

7. Choosing a Model on Independent Data and Real Cost

Every vendor in this category has published a chart showing itself in first place, which is exactly why you should not choose on vendor charts. The best independent source available right now is JevBench, a community leaderboard that describes itself as measuring decision models "independently of TypeSafe AI", scoring each system on 1,500 decisions drawn partly from a sealed set that is retired after limited use to prevent training on the test - JevBench. It rates four axes (Intelligence, Calibration, Speed and Cost) and, crucially, it runs ordinary LLMs through the same questions, which lets you see what a decision model actually trades away.

It is still one benchmark run by one small team, and its weighting choices are its own. Use it to narrow a shortlist and to calibrate your expectations, then let your own labeled data make the final call. The numbers below come from the API board as updated on October 12.

7.1 Price per token: close to a commodity already

Start with list prices, because they show how fast the category has converged. Four of the seven hosted options sit within half a cent of each other per million input tokens, and the three outliers are outliers for a reason: OpenAI's endpoint accepts a million tokens of context, and Clef and Clef-omni are larger models with video input.

Converted into decisions, those prices are small enough that the bill is rarely the deciding factor until you reach serious volume. The table below assumes a 400-token request, which is typical for a short record with two or three questions.

Two caveats change these numbers in practice. Liquid's own API bills each question as its own prompt, including its text and any images, so a request with three questions costs roughly three times the single-question figure there, while TypeSafe's design reads the state once for all questions - Liquid AI. And batching questions matters more than model choice: TypeSafe's GDPR cookbook found that putting 13 questions about one document into a single call was 12.2x cheaper and 10.0x faster than 13 separate calls, with no change in the answers - TypeSafe cookbook.

7.2 The finding vendors leave out: LLMs are still more accurate

Here is the result that matters most for an honest decision. When JevBench asks general-purpose LLMs the same questions, they score far higher on Intelligence than any decision model. OpenAI's GPT-6 Luna at low reasoning effort scores 97, against 64 for Jev and 57 for OpenAI's own decision endpoint, which runs on the same model family. Decision models are not a smarter way to make decisions. They are a much cheaper and faster way to make decisions that are good enough, with probabilities you can route on.

The reason decision models still win the overall ranking is the other two axes. JevBench's measured cost per 1,000 decisions runs from $0.017 for Liquid's d1 to $0.052 for OpenAI's decision endpoint, while the same thousand decisions cost $0.108 on GPT-6 Luna at low effort, $0.480 on DeepSeek V4.1 Flash with thinking on, and $2.18 on Qwen3.8 27B through the host JevBench used - JevBench API board. On its composite score, which weighs all four axes equally and penalizes slow or expensive systems heavily, the top hosted decision models land around 71 to 74, GPT-6 Luna at 40.5, and the larger open LLMs near zero.

Put the two charts together and the decision rule writes itself. If a wrong answer costs you more than about a tenth of a cent, and the volume is modest, an LLM call may simply be the better tool. If you make thousands or millions of these calls, or the decision sits on a latency-sensitive path, a decision model is the right default, and the uncertain cases can be escalated to an LLM as Section 9 describes. Early field reports match this: one developer told TechCrunch that Gemini was "slightly more accurate" than Jev at classifying business emails but 10 to 20 times more expensive, while a Vercel engineer reported that replacing an OpenAI Luna classifier with Jev for reviewing commands made it 5 to 18 times faster with better accuracy - TechCrunch.

7.3 Run your own evaluation in an afternoon

Microsoft's guidance for Decision-1 is the right instruction for every model here: compare "decision accuracy, including ambiguous cases, latency under expected traffic and cost per completed decision", and validate confidence calibration "on your own data" - Microsoft. You need about 200 real examples with the answer you would have wanted, which most products can pull from their own history in an hour.

The script below runs one Choice question over a labeled CSV against any SystemOne-compatible endpoint and reports accuracy, the multi-class Brier score (the mean squared error of the probabilities, where lower is better and 0 is perfect) and accuracy by confidence band. Change the base URL and model to compare vendors on identical inputs.

import csv, os, statistics, time
import httpx

BASE_URL = os.environ.get("DECISION_URL", "https://api.typesafe.ai/v1/systemone")
API_KEY = os.environ ["DECISION_API_KEY"]
MODEL = os.environ.get("DECISION_MODEL", "jev-1.13.0")
OPTIONS = {
    "billing": "Charges, invoices and refunds",
    "technical": "Bugs, outages and integrations",
    "account": "Sign-in and account access",
    "other": "None of the above",
}

rows = list(csv.DictReader(open("labeled_tickets.csv")))  # columns: text,label
correct, briers, latencies, bands = 0, [], [], {}

with httpx.Client(timeout=30) as http:
    for row in rows:
        start = time.perf_counter()
        r = http.post(BASE_URL, headers={"Authorization": f"Bearer {API_KEY}"}, json={
            "model": MODEL,
            "state": {"ticket": row ["text"]},
            "questions": {"team": {"type": "choice",
                                   "instructions": "Which team should handle `ticket`?",
                                   "criteria": OPTIONS}},
        })
        latencies.append(time.perf_counter() - start)
        data = r.json()
        data = data.get("result", data)  # Cloudflare's REST API wraps the body in "result"
        answer = data ["answers"]["team"]
        probs = answer ["probabilities"]
        hit = answer ["choice"] == row ["label"]
        correct += hit
        briers.append(sum((probs.get(k, 0) - (k == row ["label"])) ** 2 for k in OPTIONS))
        band = round(answer ["confidence"], 1)
        bands.setdefault(band, []).append(hit)

print(f"accuracy {correct / len(rows):.1%}, brier {statistics.mean(briers):.3f}, "
      f"p50 latency {statistics.median(latencies) * 1000:.0f} ms")
for band in sorted(bands):
    hits = bands [band]
    print(f"confidence ~{band}: {sum(hits) / len(hits):.0%} correct over {len(hits)} cases")

The last loop is the one that pays for the exercise. If answers at confidence 0.9 are right 98% of the time and answers at 0.5 are right 60% of the time, you have found your automation threshold, and you can estimate exactly what share of traffic will need human or LLM review. Run the same file through two or three models, multiply the review share by what a review costs you, and the cheapest model on paper is often not the cheapest in practice. Our guide to pricing an AI product against token costs shows how to fold that per-decision cost into your own pricing.

Why this matters: the scorecard at the top of this guide is a starting point, but model quality on your inputs varies more than the ranking suggests, and the independent data shows a real accuracy gap to LLMs. How to apply this: shortlist two or three models from the scorecard, run the evaluation script on 200 labeled examples, choose the model with the lowest total cost once review cost is included, and keep the script to rerun whenever a vendor ships a new version.

8. Running Decision Models on Your Own Hardware

A decision model is small, reads its input once and writes nothing, which makes it one of the few useful AI components that runs comfortably on a laptop or an edge device. That opens options the hosted APIs cannot match: no per-call fee, no network round trip, no data leaving your machine, and no dependency on a vendor that briefly ran out of capacity in its first week. The trade is that you own the hardware, the uptime and the model upgrades, and that the open models available today score lower on independent tests than the best hosted ones.

Four routes are mature enough to recommend, and they cover the range from a developer laptop to a data-center GPU. The common thread is the API: three of the four serve the same /v1/systemone shape as Jev, so code you wrote against TypeSafe can move to your own machine by changing the base URL, which is the strongest practical argument for writing to that contract in the first place.

8.1 Ollama: the fastest way to try it locally

Ollama added decision models on September 29 in version 0.35, with a local /v1/systemone endpoint that follows TypeSafe's API - Ollama. The first model offered is Nimble, a 9B decision model from Bespoke Labs fine-tuned from Qwen3.5-9B and released under Apache 2.0, which answers in under 100 ms on a MacBook Pro with an M5 Max. Ollama's Pac-Man demo averaged 91 ms per decision on that machine.

ollama pull nimble

curl http://localhost:11434/v1/systemone -d '{
  "model": "nimble",
  "state": { "ticket": "I was charged twice. Please refund the extra payment." },
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": { "billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above" }
    },
    "refund": { "type": "noul", "instructions": "Does the customer explicitly ask for a refund?" }
  }
}'

Ollama's model page lists the limits worth knowing before you build on it: up to 64 questions per request, and 2 to 26 options per Choice or Score question, far fewer than Jev's 255 - Ollama. It also works with TypeSafe's official Python SDK pointed at the local address. On independent tests, Bespoke's Nimble scores between 43.1 and 56.5 on JevBench Intelligence depending on the version measured, which places it below the hosted leaders but well within useful range for clear-cut tasks.

8.2 Liquid's d1-3B: edge devices and robots

Liquid AI released two open-weight decision models on October 7: d1-3B, which reads text and images, and d1-omni-600M, an experimental checkpoint that handles text with either images or audio - Liquid AI. Their selling point is speed on small hardware. Liquid reports d1-3B answering a single question in 8 ms on an RTX 4090, 30 ms on an Apple M5 Pro, 16 ms on a Jetson AGX Thor and 50 ms on a Jetson Orin Nano, with day-one llama.cpp support and quantized builds (GGUF and MLX) already on Hugging Face.

The license and the quality both need a careful read. Liquid's post describes the models as free to "download, fine-tune, and deploy without restrictions", but the Hugging Face model card lists Liquid's own LFM Open License v1.0 rather than Apache 2.0 - Hugging Face, so check its commercial terms against your company before shipping. On quality, Liquid reports d1-3B at 48.57 on the Decision Index, while JevBench's independent run puts its Intelligence at 22.1, the widest vendor-versus-independent gap among the models covered here. It is a strong fit for simple, high-frequency visual or audio checks on a device, and a weak fit for nuanced text judgments.

8.3 Strands Decider: small, fully open, built for agents

AWS's Strands Labs released Strands Decider 2B on October 1 under Apache 2.0, with the training data and scripts included, which makes it the most transparent model in the category - GitHub. It runs a median of about 115 ms on an RTX 3090 and about 153 ms on an M3 MacBook, and AWS reports it ranked third of 33 models in the 2B class on JevBench's accuracy and calibration. The Hugging Face organization has since added larger variants, including 12B and 26B mixture-of-experts versions built on Gemma 4.

pip install strands-decider

strands-decider ask StrandsAgents/strands-decider-2B-hobson-v19 \
  --state "Help! My payouts have been failing for 3 days!" \
  --choice "Which team should handle this?=billing,sales,retail"

Its real value is the agent integration. The Strands example runs the decider before every tool call through an InterventionHandler, asking whether the tool's arguments are grounded in what the user actually said and whether it is premature to call the tool at all, and then returns one of four typed actions: proceed, deny, confirm with a person, or guide the agent back with feedback - Strands Agents. That is a concrete, local version of the agent-monitoring pattern from Section 2, and it costs nothing per call.

8.4 Self-hosting Clef with SGLang

The largest open model in this group is Cloudflare's Clef, and it is the one to choose if you need open weights with Clef's accuracy, vision and 64K context. The model card's tested setup is a single H200 GPU, with launch commands for B200 and B300 in SGLang's cookbook, and the server answers the same /v1/systemone requests as Jev - Hugging Face. Cloudflare says native Clef support is landing in SGLang 0.5.22, with a dedicated container image available until then.

docker run --gpus all --shm-size 32g -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface --ipc=host \
  lmsysorg/sglang:dev-clef \
  sglang serve --model-path Cloudflare/clef --host 0.0.0.0 --port 30000

curl http://localhost:30000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"model":"Cloudflare/clef","state":"My Stripe integration keeps failing. Please help ASAP.","questions":{"urgency":{"type":"noul","instructions":"Does this message express urgency?"}}}'

A dedicated H200 only makes economic sense at real volume. For comparison, $1,000 buys more than four billion Clef input tokens on Workers AI, or about ten million decisions at 400 tokens each, so self-hosting Clef is mostly a choice for data control, air-gapped environments or custom fine-tunes rather than for cost. Our comparison of open-weight models to self-host covers the GPU economics in more depth.

8.5 Local latency at a glance

The chart below collects the per-question latencies each vendor reports for its open model on specific hardware. These are vendor measurements on different machines and request sizes, so read them as orders of magnitude rather than a ranking.

All of these sit at or below the hosted medians reported earlier in this guide (about 150 ms for Clef and 240 ms for Jev on JevBench), and none of them includes a network hop, which is the core argument for local deployment when latency is the constraint. The limitation is quality: none of these open models matches the best hosted ones on JevBench's sealed questions, so local works best for well-defined decisions with clear criteria, and the hosted models remain the better choice for subtle judgments.

Why this matters: local decision models make a whole class of features free at the margin and keep sensitive data on your own machines, which is the strongest form of control you can have over this layer. How to apply this: prototype with Ollama on a laptop, move to d1-3B for devices and robots, use Strands Decider when you want a fully open model wired into an agent's tool calls, and reserve self-hosted Clef for regulated or high-volume workloads where owning the GPU is justified.

9. Production Patterns That Work

Knowing the API is the easy part. The design work is deciding where a decision model sits in your system, what it is allowed to decide on its own, and what happens when it is unsure. The patterns below are the ones that recur across TypeSafe's cookbooks, Vercel's gateway features, Microsoft's and AWS's examples and early production reports, and they combine well: most real systems use three or four of them at once.

The diagram shows how they fit together in a typical request path. A decision model sees every request, code acts directly on confident answers, uncertain cases escalate to a language model or a person, and generative models only do the work that requires writing.

9.1 Confidence-gated routing

The defining pattern is to give each action its own threshold based on what a wrong answer costs. TypeSafe's voice-banking example routes anything under 0.6 confidence to a support agent, lets a balance check run at 0.6, and requires more than 0.85 before approving a transfer automatically, asking the user to confirm in between - TypeSafe. The model answers the question "what does the user want"; your code answers "is that certain enough for this particular action".

The thresholds themselves should come from data, not intuition. Run the evaluation script from Section 7 on labeled examples, find the confidence level above which answers are right as often as you need, and set the threshold there. Then revisit it whenever you change model versions, because a new release can shift the confidence distribution even when accuracy holds steady. Logging the confidence with every decision is what makes that recalibration possible.

9.2 The cascade: cheap model first, expensive model for the hard cases

The cascade is the pattern that resolves the accuracy gap from Section 7. A decision model handles every request, and only the uncertain ones go to a language model that is more accurate but costs several times more per call. If 85% of your traffic is clear-cut, you pay decision-model prices for most requests and LLM prices for the rest, and you get close to LLM accuracy overall. TypeSafe's structured-extraction cookbook applies the same idea to data extraction, using Jev to score a cheap model's output and escalate only flagged items to a reasoning model, to "get most of the quality of a big reasoning model at a fraction of the cost" - TypeSafe cookbook.

You can build the cascade in code with a single if on the confidence field, or let Vercel's gateway do it with the conditional fallback shown in Section 6.4. The gateway route is less code but bills both stages and adds both latencies on escalated requests; the code route lets you send the escalated case to a different prompt or a human, which is often what you actually want for the hardest cases. This is the same economics our guide to cutting agent costs with model routing describes for generative models, applied one layer earlier.

9.3 Speculative fan-out: ask everything at once

Because a decision model reads the state once and scores all questions in parallel, extra questions are nearly free. TypeSafe recommends putting every question your system might need into one request, including ones that only matter in some branches, and letting code ignore the irrelevant answers. Its example asks for a ticket's category, bug severity, whether reproduction steps exist and whether a refund is requested in a single call, then reads the severity only if the category turns out to be a bug - TypeSafe.

This inverts a habit from LLM development, where you ask one thing at a time to keep prompts focused and outputs parseable. With decision models, the latency of a decision tree collapses from one round trip per level to one round trip total. Brooker's measurements on his own model show why: the work is dominated by reading the state, so latency "scales very well with the number of questions" - Brooker. The GDPR cookbook's 12.2x cost saving from batching 13 questions is this pattern measured.

9.4 Guardrails and agent oversight

The pattern with the largest long-term impact may be checking agents rather than users. A decision model is cheap enough to evaluate every proposed agent action against its task before it runs, which was impractical with frontier models. The Strands example in Section 8 asks two yes/no questions before each tool call (are the arguments grounded in what the user said, and is it premature to call the tool now) and blocks, confirms or redirects accordingly. TypeSafe's guardrails cookbook does the input-and-output version for chat apps, screening every message going into and out of an LLM with one request and thresholding hazard probabilities to pass, review, block or route - TypeSafe cookbook.

Two cautions keep this pattern honest. First, decision models are themselves vulnerable to adversarial content: TypeSafe's own notes say text "written to adversarially steer the model" can move the answer - TypeSafe, so a guardrail model is one layer, never the only one. Second, oversight checks are most valuable on the actions that are hard to undo, such as payments, emails, form submissions and deletions, so concentrate your strictest thresholds there. Our guides to letting an AI agent spend money safely and securing an AI-built app before launch cover the controls that should sit around this check.

9.5 Retrieval: filter and rerank before the expensive call

Retrieval pipelines are full of small relevance judgments, which makes them a natural fit. TypeSafe's reranking cookbook took 30-passage keyword shortlists for 40 legal queries and used one question per query-passage pair to raise top-1 accuracy from 5% to 18% and top-10 accuracy from 38% to 62% - TypeSafe cookbook. The same approach works earlier in the pipeline as a filter that decides which passages reach the answering model at all, which cuts the tokens you pay the generative model for.

Liquid's demos show a related use inside coding agents. Its context-compaction example has d1 read each tool output in an agent's session and decide whether to keep, trim or drop it for the next step, removing 52% of the tokens while keeping every output the task needed - Liquid AI. Given how much agent cost comes from carrying old context forward, a cheap keep-or-drop decision is one of the highest-return places to put this category, and it pairs well with prompt caching on the generative side.

9.6 Composite scoring: many small judgments, weights in code

When a decision depends on several factors, resist the urge to ask one big question. TypeSafe's composite-scoring pattern breaks a judgment such as resume screening into independent Score questions, normalizes each to 0 to 1, and combines them with weights you control in code, which "gives you visibility into how exactly the final score is being calculated" - TypeSafe. The same structure fits lead scoring, content moderation with several policies, or vendor risk assessment.

The benefit is less about accuracy than about control. When the ranking looks wrong, you can see which dimension drove it and change one weight, rather than rewriting a prompt and hoping. It also keeps the parts that are not judgment, such as thresholds, arithmetic and business rules, in deterministic code where they belong.

Why this matters: these patterns are what turn a cheap model into a reliable system; used naively, a decision model is just a faster way to be wrong. How to apply this: start with confidence gating and a cascade on one high-volume decision, batch every related question into the same call, and add an action check in front of any agent tool that can spend money or send messages. Measure the escalation rate weekly, because it is the number that tells you whether the model, the thresholds or the questions need work.

10. Where Decision Models Fail

Every model category has a failure profile, and this one is unusually well documented because TypeSafe published its own. Its "jaggedness" page for Jev 1.13, last reviewed October 2, lists nine known failure modes with a workaround for each, and opens with a blunt summary: the model "may struggle with tasks that require additional levels of indirection", "can be quite literal", and "struggles with tasks that require numeric precision" - TypeSafe. Most of these apply to every decision model, because they follow from the design rather than from one vendor's training.

The table condenses the failure modes that matter most in production and what to do instead. Treat it as a checklist for writing questions, because nearly every item is fixed by changing the question or the code around it rather than by changing models.

10.1 Literal reading and the arithmetic trap

The literal-reading problem catches people who are used to LLMs that guess at intent. TypeSafe's advice is precise: "when you look at a wrong answer and find yourself explaining what you really meant, that explanation is the missing half of the instruction." If the question is "is this customer high value?", the model has no idea what your business means by high value; write "has the customer spent more than the threshold in lifetime_spend" and pass the threshold as data, or better, compute that comparison in code and ask the model only about the parts that are genuine judgments.

The arithmetic trap is the same lesson from the other side. Decision models do not calculate; they recognize the shape of an answer. TypeSafe's counting example replaces "how many of these items are fruits" with one yes/no question per item and a sum in code, which turns an unreliable count into a series of easy judgments. The broader rule from its documentation is worth pinning above your desk: avoid "asking the model something code can compute exactly" and "hiding several judgments inside one question". Microsoft's announcement makes a related point about consistency, reporting that Decision-1 changed its answer on 1.3% of reworded requests on average - Microsoft, which is low but not zero, and one more reason to keep exact logic out of the model.

10.2 Context rot and hostile input

A decision model's accuracy degrades as you add irrelevant material to the state, which TypeSafe calls context rot. This runs against the LLM habit of including everything just in case. The fix is a retrieval or filtering step before the decision: select the fields and passages the question needs, and if you cannot filter in code, use a cheap Noul to filter for relevance first. As Section 5 showed, the same step also cuts latency, since a 16,000-token state takes about ten times as long as an 800-token one on Clef.

The hostile-input problem is more serious for anything user-facing. The state is data, but a model reading it can still be steered by text crafted to argue for its own classification, such as a support message that says "this is not spam" or a scraped page that tells the classifier it is safe. Better Stack's test opened, fittingly, with a prompt injection found in its own YouTube comments. Write precise criteria, test with adversarial examples before deploying, and never let a single decision model be the only control in front of an irreversible action.

10.3 Calibrated is not the same as correct

The probabilities are the product, so it is worth being exact about what they promise. A calibrated model is right about 80% of the time across the answers it gives 0.8, which means one in five of those answers is wrong by design. Calibration is also measured on someone's benchmark distribution, not yours; when your inputs differ from the training data, the probabilities can drift in either direction. And confidence on a Choice measures how concentrated the distribution is, which Ollama's model page states plainly "isn't the chance that the answer is right".

The forced-choice problem compounds this. A model given three options will distribute its probability across those three even if the true answer is a fourth, so an out-of-scope input can come back with apparently decent confidence on a wrong category. Cloudflare's own CLINC150 result, a test designed with out-of-scope requests, shows how large that effect can be: Clef-flash scored 66.77 against Clef's 97.43. Always include an explicit "other" or "none of these" option, and track how often it is chosen.

10.4 Vendor numbers versus independent ones

The category's marketing is unusually number-heavy, and several of the most quoted figures deserve a discount. TypeSafe's "193.6x faster" and "444.6x cheaper" come from its own workflow evaluations, and the launch post itself says those multiples are likely "on the higher end of real world gains", that the authors built the evaluations, and that they "can't prove" the pricing is not subsidized - TypeSafe. That candor is to TypeSafe's credit, and it is also a reason not to put those numbers in your business case.

The gaps between vendor and independent results are large enough to change decisions. Cloudflare's chart has Clef ahead of Jev on the Decision Index, while JevBench's sealed questions put Jev ahead on Intelligence (63.6 to 59.1). Liquid reports d1-3B at 48.57 on the Decision Index, while JevBench measures its Intelligence at 22.1. Microsoft reports the highest accuracy in its own 36-benchmark comparison, while JevBench places Decision-1 sixth on its composite. None of this means a vendor is lying; benchmarks measure different things on different question sets. It means the only result that settles a choice is one on your own data, which is why Section 7 ends with a script rather than a winner.

10.5 The risks of a four-week-old market

The final category of failure is not technical. Every product in this guide is weeks old: OpenAI's endpoint is a public beta, Microsoft's is a public preview, Clef-flash changed its price and context window eight days after launch, and the AI SDK renamed its function from evaluate to decide within a month. Model aliases move, and TypeSafe's API returned overload errors in its first week. Any of these vendors could reprice, rename or retire what you depend on, as builders are seeing this month with OpenAI's model retirements.

The mitigations are the ones this guide has repeated, and they cost little. Pin model versions and upgrade deliberately. Write to the shared /v1/systemone contract so a second vendor is a configuration change away. Keep a fallback model configured for errors as well as for low confidence. Keep your labeled evaluation set and rerun it on every upgrade. And keep a deterministic default for every decision, so that if the model is unavailable your product degrades to a safe behavior rather than stopping.

Why this matters: most decision-model failures in production will come from questions that ask for arithmetic, states stuffed with irrelevant text, missing "other" options and trust in vendor benchmarks, not from the models being weak. How to apply this: review every question against the table above before launch, run an adversarial test set, and treat each vendor as replaceable from the start.

11. What Comes Next

Predicting a four-week-old category is risky, so it helps to reason from the forces in Section 2 rather than from the announcements. The cost of a decision is set by the cost of reading the input, the backbones are open and improving every few months, the API contract is shared, and agents keep generating more small judgments per task. Each of those points in a direction you can plan around even if individual vendors rise or fall.

11.1 Prices fall toward the cost of reading

The first consequence is continued price compression. Four hosted models already sit within half a cent per million input tokens of each other, Cloudflare cut Clef-flash by more than half eight days after launch, and the underlying work (a single prefill pass over the input) is the cheapest operation in language-model inference. When a product's main cost is a commodity input and several providers can supply it, the price tends toward the cost of that input. TypeSafe's own launch post anticipates this, saying it expects prices to fall over time.

That makes the price per decision a weak basis for choosing a vendor and a strong basis for redesigning features. Ask which checks you have avoided because an LLM call per row, per email or per agent step was too expensive, and assume those checks will cost close to nothing within a year. The Jevons argument from Section 2 says the winners will be the products that use far more decisions, not the ones that save the most on the decisions they already make.

11.2 Every modality, and models tuned to one company's data

The second direction is breadth. In five weeks the category went from text only to images (Clef, d1), video (Clef) and audio (Clef-omni, d1-omni-600M), and TechCrunch reported that TypeSafe itself plans more versions of Jev in new modalities - TechCrunch. Liquid's demos already include visual inspection on factory lines, where d1 sorted good and defective parts with 85 to 97% accuracy without being trained for the task - Liquid AI, and Microsoft lists robotics and computer-use navigation among its intended uses. A decision model that reads a camera frame and picks an action in tens of milliseconds is a very different building block from a chatbot.

The third direction is customization. Cloudflare is offering hands-on fine-tuning of Clef now and plans a self-serve platform that captures traffic through its AI Gateway, generates rollouts, trains and redeploys on Workers AI - Cloudflare. That is the old custom-classifier workflow reborn with a general model as the starting point: you no longer need thousands of labels to get started, but your own labeled decisions become the way to beat the general model on your domain. Teams that start logging decisions, confidence and eventual outcomes today will have the training data when this becomes routine.

11.3 The decision layer becomes invisible infrastructure

The fourth direction is absorption. Vercel's eve framework already uses Jev by default for model selection and tool approvals, AWS wires its decider into the Strands agent SDK, OpenRouter sells a Jev-based router, and Microsoft says future versions of Decision-1 will be rebased on OpenAI and Microsoft's own MAI models - Microsoft. Much of the category's volume will soon run inside frameworks and gateways that choose a decision model on the developer's behalf, the way embedding models quietly became part of every retrieval stack.

For builders, that cuts both ways. It lowers the effort of adopting decision models to nearly zero, and it also makes it easy to depend on one without knowing which, at what version, with what data terms. The habits from this guide (pin versions, log the model that answered, keep an evaluation set, write to the shared contract) are what keep that dependency visible and under your control. If you are thinking about the broader shape of an AI product's stack, our breakdown of the AI-native company tech stack shows where a decision layer fits among the other components.

11.4 Where the value accrues

The $7.5 billion valuation is a bet that quality and calibration data, not architecture, will separate the leaders from the dozens of copies, and the independent data so far is consistent with that bet: the architecture spread to 155 open systems in a month, but the gap between the best and the median remains wide. It is also a bet that the market stays large while prices fall, which depends on builders finding many more places to put cheap judgments.

For a product company, the more durable value sits one level up. A decision model makes a judgment cheap; it does not decide which judgments matter, what a wrong answer costs, or what the product should do with the answer. Those are product and domain questions, and they are where the advantage of using this category well will come from. That is the same pattern our guide to what software is left to build describes for AI more broadly: when an input gets cheap, the opportunity moves to the people who apply it.

Why this matters: the parts of this market that will change fastest (prices, rankings, model names) are the parts least worth optimizing for today, while the parts that compound (your labeled decisions, your question designs, your confidence thresholds) belong to you whichever vendor you use. How to apply this: start logging every decision with its inputs, confidence, model version and eventual outcome now, so that you can fine-tune, switch vendors or move to a local model later on evidence rather than on announcements.

12. Conclusion: A Decision Framework

Decision models are a genuinely new building block in the AI stack, and the speed of the past four weeks shows that the industry agrees. They are also narrower than the excitement suggests: they score fixed options, they are less accurate than general LLMs on the same questions, and they fail predictably when asked to count, compare dates, read through irrelevant context or resist crafted input. Used where they fit, with confidence gating and a cascade to a stronger model, they make a large class of product features cheap and fast enough to run on every request.

The choice of model matters less than the design around it, but a few defaults hold for most teams, and the table below condenses the guide into a starting point for each common situation. Treat it as a shortlist to test, not a verdict: the evaluation in Section 7 should make the final call.

Whichever you pick, the sequence that works is the same. Find the decisions in your product that currently go to an LLM, rewrite each as a Choice, Score or yes/no question with an explicit "other" option, run 200 labeled examples through two or three models with the script from Section 7, pin the winner's version, gate actions on confidence with thresholds set from your data, and escalate the uncertain cases to an LLM or a person. Log every answer with its confidence and model version, and rerun the evaluation before every upgrade.

The most important habit is the one that protects you from a market this young: write to the shared contract and keep the model name in configuration. Then the next price cut, the next better model or the next outage is a one-line change instead of a rewrite, and the decision layer of your product stays yours.

This guide reflects the decision model landscape as of October 13, 2026. Prices, model versions, limits and benchmark rankings in this category have changed weekly since September, so verify current details on each vendor's documentation before you build.