Reef puts a learning loop between an agent and its model provider

The Human-Agent-Society group has published Reef, an Apache-2.0 Python package that hosts an inference endpoint and turns the traffic through it into training signal for the model or the agent harness behind that endpoint. Reef went public on 31 August 2026 and has picked up 353 stars in its first four days on GitHub.

What it does

Reef serves OpenAI- and Anthropic-compatible endpoints at /v1/chat/completions and /v1/messages, so any client that already talks to those providers can switch to Reef with a token change. Every response carries an x-reef-agent-record-id header, and a later call to /reef/report can attach a numeric score, free text, or a structured object to one or more of those receipts. The scenario a request belongs to is set by an x-reef-scenario header, and a new name creates a scenario using the deployment's configured recipe.

Two surfaces to improve

Reef can update model weights or an agent harness, chosen by the recipe attached to the scenario. Weight updates run one of the reinforcement-learning cookbook recipes, which include SAO, an OpenClaw-derived recipe, and a test-time-training recipe. The harness_evolve recipe changes a tree of rules, skills, configuration, prompts, and extensions instead of weights, and does not require a GPU. In both cases Reef evaluates the candidate against the current version on the configured tasks and publishes it only when the candidate scores better.

How it fits with what teams already run

The repository compares Reef with two things a team may already have. An inference engine such as vLLM or SGLang serves traffic but does not train, and an RL framework such as slime or veRL trains but does not serve. Reef bundles both, and adds version management, live updates that do not interrupt serving, and the harness path that neither category offers.

What is included

The package is reef-infra on PyPI. The repository carries a cookbook of five recipes, an example that starts an SAO deployment against Qwen2.5-1.5B-Instruct, and a reef-pi command that installs an evolvable harness from a running Reef server the same way most coding agents install themselves. Reef uses Git LFS locally for its artifact repositories, and the training examples need a GPU that matches the recipe's requirement; the harness path does not.

The problem Reef targets appears after the first agent is released. A team collects task outcomes, sees which prompts and models handle which cases well, and then has to build its own pipeline to feed that signal back into anything. Reef offers a single server that captures the receipt at inference time, matches feedback back to it, batches eligible reports, retrains, evaluates, and promotes. The trade-off is a new dependency: the agent has to send its inference through this endpoint and grade its own output, which is a real production system to run and monitor. The harness recipe lowers the entry cost, because a team can evolve prompts, skills, and configuration without any GPU, and later swap in a weight-training recipe once there is enough labelled traffic. Four days of adoption shows the problem is real, though the project itself has no history yet.

Source

Primary source: Human-Agent-Society/reef on GitHub.