Hugging Face adds native support for decision models in Transformers

The library that most AI engineers use to work with open models has added a new kind of model. Hugging Face said its Transformers library now has native support for decision models. A decision model does not generate text. It takes a short list of choices and returns a probability for each one.

Four Hugging Face employees wrote the post: Aritra Roy Gosthipaty, Lysandre, Pedro Cuenca and Tom Aarsen. The release defines three primitives a decision model can answer with. A choice picks one option from a short list. A score rates an input on a numbered scale. A noul is a yes-no answer with a probability on each side. Each one returns a probability distribution, not a sentence.

What changes for an engineer

The new API is a single pipeline call. Hugging Face lists transformers support and a /v1/systemone HTTP endpoint for serving. It is also compatible with llama.cpp. The company offers four example models in its own hub. They are GEV-26B-Decide, JEV-9B, Kev-4B-GGUF and ModernBERT variants. Any model tagged decision-model on the Hugging Face hub works with the new API. Fine-tuning is supported through the standard Trainer. Hugging Face says support in TRL is coming.

Why the shape matters

A chat model asked to pick a department returns a sentence. The sentence has to be parsed. The parser may fail if the model rewords the answer, drops a bracket or adds a preamble. A decision model answers with a defined set of keys and a probability for each. The output shape is guaranteed. The example in the post routes a customer message to "billing", "technical" or "sales". The probabilities are 0.998, 0.001 and 0.001. The team using it does not have to write JSON-schema prompts or a parser.

The score primitive is useful as a grader. A language model can rate a response on a 1 to 5 scale. The same job can be a decision model. It returns a probability for each number and reports the expected value. The post shows an expected value of 1.7 on a 0-to-2 scale. A noul primitive is a direct replacement for a yes-or-no classifier.

What this does not say

Hugging Face does not state an accuracy or a speed figure against any chat model with a strict JSON schema. The post gives no public benchmark number. The confidence formula and the softmax temperature appear only as code. There are no measured outputs. All four of the named example models come from Hugging Face's own partners. A team that wants an open benchmark against their own data will have to build one.

A team doing AI engineering today has at least one decision point in every agent they run. Which tool to call. Which team to route to. Whether a reply is ready to send. How good a model's output is. Each of those is a classification job the team currently asks a chat model to do. The gain of this release is that a decision model can do it directly. The cost is a second kind of model to manage in production. It has its own training data, its own evals and its own vendor choice. Replace one JSON-schema prompt in your code with a decision pipeline. Measure the two side by side on your own data. That is the fastest way to see if the trade is worth it.

Source

• Decision Models in Hugging Face, Hugging Face, Aritra Roy Gosthipaty, Lysandre, Pedro Cuenca and Tom Aarsen