Together AI's HybridAgent paper reports 35 to 55 percent fewer agent tokens

An AI agent that reads, thinks and acts in words pays for every one of those words in tokens. A paper on arXiv argues the model can do the thinking in a hidden state and send 35 to 55 percent fewer tokens through the API.

Hybrid Reasoning Agents, by Jiaru Zou at Stanford University and seven co-authors from Stanford, the University of Illinois Urbana-Champaign, Together AI and Meta, reports an architecture that lets a model switch between thinking in a hidden state and acting in text. The paper is on arXiv as 2610.18861 under a CC BY 4.0 licence, with code and data at togethercomputer/HybridAgent on GitHub.

Switching between a hidden state and text

Today an AI agent writes out its reasoning, then calls a tool, then reads the tool's reply, then writes more reasoning. Every one of those steps is text, and the model is paid by the token for all of it. HybridAgent adds a small piece of code, called HybridLink, that reads the model's last-layer hidden state and feeds it back as an input embedding. The agent can enter this latent mode when it is only thinking, then exit it and emit text when it is time to call a tool. A "continuous working memory" carries the hidden state across turns so the context survives a tool call.

Training is in two stages. HybridSFT, a supervised fine-tune, teaches the base model to work with HybridLink. HybridRL, a reinforcement step, teaches the model when to think silently and when to speak.

10.4 percent higher scores, 35 to 55 percent fewer tokens

The authors evaluate three model sizes (4B, 8B and 14B) across 11 benchmarks spanning mathematics, science, medicine, search and agent tasks, including GAIA, WebWalker, Humanity's Last Exam, SealQA and DSQA. Against the frontier agents and agent frameworks it tests, the paper reports HybridAgent at 10.4 percent higher average scores, 35.5 to 54.7 percent token reduction, 1.8 to 2.6 times faster inference and 2.7 to 3.7 times lower cost. The agentic-reasoning table shows the 14B model reaching 52.4 average on GAIA against 43.7 for ARPO, the next-best 14B method the paper tests, and 49.5 on WebWalker against 40.5. The agentic reasoning numbers are averages over five independent runs with reported 95 percent confidence intervals of plus or minus 0.017 in accuracy.

The method draws on earlier latent-reasoning work the paper names, including Coconut and SoftThinking, which it also tests at 8B and 14B scales. The paper argues those earlier methods cannot call tools, which is why HybridAgent is built around a switchable mode rather than a single latent one.

The figures are the authors' own. Nobody independent has run these numbers yet, and the paper's eight authors share Together AI and Stanford affiliations in some combination, so the result is a measurement from a group that has a reason to find a high number. The code is open, so another lab can check whether the gap holds on its own agents and its own cost model.

For a team running an AI agent in production, the paper is a reason to look at how many tokens a loop spends on silent thinking between tool calls, and whether a hidden-state carry would cut that bill. The architecture is a change to the model's post-training, so adopting it means a fine-tune rather than a prompt change.

Source

Hybrid Reasoning Agents on arXiv by Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He and James Zou. Code at togethercomputer/HybridAgent.