OpenAI's Decisions API ships generally available, running 10x faster than Luna: Reveneau

Why it mattersTeams running agents can now move the small routing and classification calls off a full language model onto a focused decision endpoint, which is where latency and cost usually pile up in a production workflow.

An AI agent is a stack of tiny decisions, and most of them do not need the full language model answering them. OpenAI moved its Decisions API out of public beta and into general availability on 22 October, built on GPT-6 Luna but focused on questions with bounded answers.

The company says the Decisions API runs up to 10 times faster than GPT-6 Luna called through the Responses API, with that claim covering text, image and structured JSON inputs. The post is OpenAI's own and does not include an independent benchmark. The service costs $0.10 per million input tokens, with cached input at $0.01 per million tokens, and OpenAI says there are no output-token charges.

What the GA release adds

Since public beta, OpenAI added a native GPT-Live integration so a voice agent can call Decisions in-session, prompt caching for reused context, and inference residency in the US and EU. Developers can now pass native JSON inputs, include images in question instructions, and ask up to 400 questions in a single request, double the previous limit. The company also says it has added capacity in several Asian regions to bring latency down there.

The API still returns one of three answer shapes OpenAI defined in the beta: a predicate with a probability from 0 to 1, a choice from a predefined set with per-choice probabilities and confidence, and a score that is a probability-weighted average of ordered rubric levels.

What customers reported

OpenAI's post names five customers and their claimed results. Box says the Decisions API ran about 5 times faster than the Responses API on 30 certificate of insurance routing cases with the same accuracy. Datadog says it uses the API to score the risk of agent-generated shell commands before running them, and that the API runs 40 times faster at the 95th-percentile tail than the frontier model it compared against, with comparable accuracy. Ramp says the API attributes its AI costs about 4 times faster and 30% cheaper than calling GPT-6 Luna, without a drop in accuracy. Sierra says it uses the API during live customer conversations to pick the next action, at under 280 milliseconds at the 95th-percentile tail, about 3 times faster and 90% lower cost per call than its previous approach. Speak says the Decisions API in GPT-Live reduced its time to next action from 1.4 seconds to 0.13 seconds. All five figures are the customers' own numbers.

Where it fits

TechCrunch, which covered the launch, reported that OpenAI's product manager Umesh Handa said the model is "just plain old Luna" with its inference reconfigured to run faster and cheaper, rather than a separately trained decision model. The piece notes OpenAI's Decisions API ranks eighth on the AIM System One Index and 14th on the Artificial Analysis Decision Index, with TypeSafe's Jev model at seventh and third on the two lists.

Who changes what tomorrow

An agent engineer who has been routing with a general-purpose model has a specific swap to try: the small classifications, routing selections, and risk scores that run on every turn can move out of the Responses API and into Decisions with a defined question and a bounded answer set. OpenAI publishes a Decisions Playground and a developer guide for the first request. The decision whether to adopt it rests on measuring your own task against your current approach.

Source

Build more responsive apps with the Decisions API, from OpenAI. Reporting also by TechCrunch at OpenAI makes its Jev competitor widely available.

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.