Microsoft used database checks as rewards and tripled an agent's clean-run rate

A failed agent run can leave the wrong record in a database. That mistake is hard to catch. It is harder to teach the model to stop doing it.

Microsoft Research used the database check as the training reward. The share of agent tasks a model passes on every one of 20 runs tripled. Tuhin Kundu wrote it up on the Hugging Face blog on 22 October 2026. The method uses Microsoft's own ThinkingBox benchmark. The benchmark has 507 workflows across retail, auto insurance, travel, neobank and consulting. Each one has a clean database. Executable checks compare the final state against the required end state.

What changed on the sealed benchmark

Microsoft fine-tuned Qwen3.8-27B with GRPO. The run was 50 steps on 187 selected training tasks. The team then evaluated the checkpoint on the full 507-task benchmark. The checkpoint had never trained on those tasks. The base model passed every one of 20 attempts on 38 tasks (7.5 percent). The fine-tuned model passed every one of 20 attempts on 107 tasks (21.1 percent). Pass@1 rose from 51.7 to 60.8 percent. Pass@20 held near flat.

Repeatable, not broader

The gains sat in the share of tasks that run cleanly every time. Pass@20 declined by 7 tasks overall. The lift was uneven across domains. Retail gained 24.5 points on observed 20 of 20. Auto insurance lost 7.0 points of pass@20. Microsoft argues that pass@1 alone is the wrong column to read. Observed 20 of 20 is what matches production reliability.

Which tasks to train on

Difficulty is policy-relative. Microsoft probed the base model against candidate tasks. The team built a Beta posterior with a Jeffreys prior. The pool kept only tasks the model passed sometimes and failed sometimes. Tasks the base model already solved were dropped. Tasks it never solved were also dropped. The final pool was 187 tasks selected from a larger candidate set. The reward is binary. A trajectory passes if the final database state matches, and fails otherwise. Nothing in the reward looks at the steps the agent took.

Cost per dependable task

Microsoft also prices the result. Count only tasks the model passes on every one of 20 runs. Cost per such task was $24.36 on the base model. The fine-tuned checkpoint brings it to $8.00. That is a 67.2 percent cut. At that price the fine-tune reaches the Pareto frontier the earlier ThinkingBox paper drew. GPT-5.4 sat at $6.80, GPT-6 Astra at $7.45 and Claude Opus 5.5 at $7.80 on that chart. The dollar figures are an inference-cost comparison, not an invoice.

The training stack is released as microsoft/thinkingbox-training on GitHub. The repository holds the preparation script, the Verl patch and the vLLM compatibility proxy. Microsoft asks teams to keep training-side and evaluation-side interpretation identical. The reasoning extraction, tool-call parsing and task state must be handled the same way. Otherwise training optimises for a different agent than evaluation measures.

A reward that only checks the final state is strict in the way a team should want. The agent cannot earn credit by writing a nice summary. It cannot earn credit by stopping with the correct message. If the row is wrong, the run fails. For a company building agents that touch customer records, the useful part is not the Qwen3.8-27B checkpoint. It is the loop that lets a team train on the same check that runs in production. Curate tasks that produce mixed outcomes under the current policy. Grade on terminal state. Treat the clean-run rate as a design target instead of the single-attempt score.

Source

• Hugging Face blog post, by Tuhin Kundu (Microsoft): Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox
• Training code: microsoft/thinkingbox-training
• Earlier benchmark: microsoft/ThinkingBox-Bench