Single agent or multi-agent: choosing an architecture

The appeal of multi-agent design is intuitive. Break a hard job into roles, give each role an agent, let them collaborate. It copies how human teams work and it looks impressive in a demo.

In production it frequently disappoints, and the reasons are worth understanding before committing.

What each additional agent actually costs

Every agent you add brings its own evaluation problem. It needs its own definition of correct and its own set of cases, because knowing the overall system is wrong does not tell you which agent caused it. Two agents mean two evaluation sets plus a third for the combined behavior.

Debugging gets harder in a specific way. With one agent, a bad result has one reasoning trace to inspect. With several, you first have to work out where the failure originated, and the answer is often that no single step was clearly wrong: each was slightly imprecise and the errors accumulated.

Cost and latency add up. Agents that communicate do so through model calls, so coordination is not free. A five-agent system can make several times the calls of a single agent for the same task, and the user waits for all of them in sequence.

Permissions multiply. Each agent needs its own scope, and the combination needs thinking about, because an agent that can request work from another agent may indirectly reach capabilities it was not granted.

The characteristic failure mode

Multi-agent systems fail in a recognizable way: information degrades as it passes between agents.

Agent A produces a summary that is broadly right but drops a qualifier. Agent B, seeing only that summary, reasons confidently on incomplete information. Agent C acts on B's conclusion. No individual step made an obvious mistake, and the final output is wrong with no visible error at any step.

This is why loose, conversational handoffs are risky. Each time information passes between agents is an opportunity to lose the detail that mattered, and nothing in the system notices.

Start with one agent and better tools

Before splitting, fully try the simpler option, because most problems that look like they need multiple agents are one agent working with poor tools or information.

If the agent is confused about which capability to use, the tool descriptions are probably unclear or the tools overlap. If it lacks information, it likely needs a better retrieval tool rather than a research agent. If it is doing too much, narrowing its job usually works better than adding another agent. If the process has fixed stages, a workflow with agentic steps inside it is almost certainly the better design.

One agent with a small set of well-described tools handles a remarkable range of work, and it remains something you can evaluate, debug, and reason about.

When splitting genuinely helps

There are real reasons to use more than one agent, and they share a property: something about the problem, rather than the aesthetics of the design, forces the separation.

Genuinely separate domains. When two parts of a task need substantially different context and expertise, and combining them would mean an instruction set that contradicts itself, separating is reasonable.

Permission isolation. This is the strongest reason. When one part of the work must read untrusted content and another must hold real power, keeping them in separate components with separate permissions is not complexity for its own sake, it is the correct security boundary.

Independent verification. When a mistake is expensive, having a separate agent check the first agent's work against the source can catch errors the original will not catch in itself. This is a genuine benefit, though it costs an extra call and needs its own evaluation.

Real parallelism. When several independent subtasks can run at once and the results are combined mechanically rather than through negotiation, parallel agents can reduce latency meaningfully.

Design handoffs like interfaces, not conversations

When you do split, how you handle the boundaries determines whether it works.

Pass structured data rather than prose, so nothing depends on one agent correctly interpreting another's phrasing. Validate at each boundary, so a malformed or implausible handoff is caught immediately rather than spreading to later steps. Preserve provenance, so a later agent can tell what was directly observed and what was inferred, which is exactly the distinction that gets lost in summarization. Give one component responsibility for the overall flow, rather than hoping several agents reach agreement through discussion.

Done this way, a multi-agent system starts to look like ordinary software architecture: components with clear responsibilities, defined interfaces, and validation between them. That resemblance is a good sign.

A reasonable path

Build one agent with a narrow job and good tools. Evaluate it properly and find out where it actually fails. If the failures come from tool quality, scope, or retrieval, fix those, since they are cheaper fixes than adding a component.

Split only when you can name the specific reason and it is one of the ones above. Then evaluate each part separately as well as the whole, so you can tell which one is responsible when the system is wrong.

A note on frameworks

The same restraint applies to the tooling around agents. Frameworks genuinely help: they handle tool calling, retries, and state so you do not rewrite that each time.

They also change quickly and make design choices for you that are awkward to reverse. A framework that reorganizes its abstractions between versions can turn a routine upgrade into a rewrite, and an unusual requirement can be harder to satisfy inside a framework's assumptions than outside them.

The practical position is to understand what your agent does without one before adopting one. Build the first version directly against the model API, so you know what a tool call, a retry, and a state transition actually involve in your system. Then adopt a framework because it removes work you understand, not because it hides work you never learned. That way the framework is a convenience you could replace, rather than the only thing that makes the system work.

The teams releasing useful agent systems today are mostly running simpler architectures than public discussion suggests. Simpler designs are the ones that keep working once they reach production.

A worked example: splitting for the right reason

A team building a system to answer questions about internal financial reports started with one agent that read a document, searched for relevant figures, and answered in text. Accuracy on straightforward questions was good, but on any question involving a calculation across two documents, the agent occasionally reported a number that was internally consistent but did not match either source.

The temptation was to add a second agent whose entire job was to verify: given the first agent's answer and the documents it cited, confirm that every figure in the answer actually appears in the cited source, and flag anything that does not. This is exactly the independent verification case described above, because the mistake was expensive (a wrong financial figure reaching a decision-maker) and checkable against a source that already existed. The verification agent did not need the same broad reasoning as the first agent. It needed one narrow job: compare stated figures to source figures and say whether they match. That narrowness is what made it worth adding, because it kept its own evaluation simple even while the first agent's job stayed complex.

Contrast this with a team on the same project that considered adding a separate agent purely to "make the writing sound more polished." That is not a case of genuinely separate domains or isolated permissions. It is a formatting preference that a well-written prompt on the existing agent could handle directly, and splitting for it would have added a coordination step and a second evaluation burden for no benefit that a prompt change could not deliver more cheaply.

How the number of agents should track team size, not ambition

A detail that is often missed in the enthusiasm for multi-agent design: every agent in a system needs someone who understands its specific job well enough to evaluate it, debug it, and notice when it silently degrades. A five-agent system built and maintained by two people means each person is genuinely responsible for two and a half agents' worth of behavior, and that responsibility does not compress just because the agents are running in the same codebase.

This is worth stating plainly during planning, because a design review that asks "does this architecture make sense" without also asking "does our team have the capacity to own this many independently evaluated components" will approve architectures that are sound on a whiteboard and unsustainable in practice. A team of three maintaining one well-tooled agent has a much easier ongoing job than the same team maintaining four agents that each work most of the time.

Recognizing degraded-information failures before they reach a customer

Because the information-degradation failure mode described above produces no visible error at any single step, catching it requires deliberately comparing what an early agent produced against what a later agent assumed, rather than only checking the final output.

A practical technique is to have the last component in a chain restate, in its own output, the specific facts it used from earlier steps, not just its conclusion. If agent A found that a shipment left the warehouse on a Tuesday, and agent C's final answer to the customer restates that date explicitly, a reviewer or an automated check can compare that restated date against the original record directly, rather than trying to infer whether C's reasoning silently dropped or altered it. This adds a small amount of verbosity to the intermediate output, and it is one of the more effective inexpensive defenses against a failure mode that is otherwise nearly invisible until a customer complains.

When the coordination itself is the product

There is one category of multi-agent system worth naming separately, because it does not fit neatly into the "add a second agent for a specific reason" framing above: systems where several agents genuinely need to negotiate, such as one representing a buyer's constraints and one representing a seller's, arriving at an outcome neither could produce with fixed logic alone.

These are rarer in practice than the current volume of discussion about multi-agent systems would suggest, and they deserve real scrutiny before being built, because the evaluation problem is at its hardest here: correctness is not a single agent being right, it is the negotiation reaching an outcome both sides would accept, which is much harder to define in writing than the more common cases described earlier in this guide. If your problem genuinely fits this shape, expect the evaluation and testing work to be proportionally larger than for any of the architectures described above, not smaller.