Google Research released Harness2, a tool for refining an AI agent's harness: Reveneau

Why it mattersAn AI engineer who tunes an agent's harness by hand gets a published method that proposes edits, measures their effect on the same task, and composes a better harness without changing model weights.

An AI agent that works on today's task often breaks on tomorrow's, and the fix is usually hours of editing the system prompt and tool guidance by hand. Google Research published Harness2, an open-source framework that does that editing for the agent, one task at a time, and reports gains of up to 11.8 percentage points on three professional benchmarks.

The repository is on the google-research GitHub organisation under Apache 2.0, Python, and has pulled 12,097 stars and 1,103 forks since the first commit. A Hacker News post for the project reached 73 points.

What the framework does

Harness2 keeps the model weights fixed and treats the harness around the model as the thing to tune. The authors call out eight editable components: the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins. These control how the agent builds its context, calls its tools, delegates to subagents and behaves inside the tool loop.

The method runs two levels of refinement at the same time. Task-level recursion proposes edits to the harness, probes the effect of each edit on the current task, and composes a better harness from the behavioural contrasts between runs. The paper calls this Propose, Probe, Compose. Harnessing-level recursion keeps the adopted harness and the lessons it produced across tasks, so the system gets better at refining future harnesses.

The numbers

Across three benchmarks and four model and harness pairs, one Harness2 step improved GLM-5 and Gemini 3.5 Flash over their base harnesses. The authors report gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. The gains grow with the task-level recursion budget. JobBench covers professional work, LAB covers legal work, and WorkBuddy-Bench covers office, web and code tasks.

The release ships two modes: parallel recursion runs K candidate harnesses in parallel and composes refinements from their traces, and sequential recursion carries one refined harness through K successive Propose, Probe, Compose steps. The parallel mode explores wider and the sequential mode refines deeper. The authors describe both and name the trade in the paper, arXiv:2610.14224.

Caveats worth stating

The repository is marked "not an officially supported Google product" in the disclaimer, which is the usual label on a research release. It supports two executors, OpenCode and Codex, which read the harness as a configuration and run each task's steps. Executor network access is unrestricted and Codex runs with its sandbox off, so a benchmark run can reach the live rubric unless the operator blocks it first. The authors say so and tell you to configure network restrictions before any grading pass.

For an AI engineer who already tunes an agent's prompt and tools by hand on each new task, Harness2 is a published method with a reproducible setup, three benchmark harnesses, and two models under test. The code, the paper and the data are all out, which is the baseline for anyone wanting to repeat the gains on a different model.

Source

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.