Designing multi-agent systems that don't fall apart

When to use multi-agent, and when not to

Multi-agent architecture has two legitimate use cases: parallelism and specialisation. Parallelism means breaking a task into independent subtasks that can run simultaneously — processing ten documents at once rather than ten in sequence. Specialisation means having agents with specific roles that handle particular types of reasoning better than a generalist agent would. If your use case fits neither of these, a single agent with a well-designed prompt is almost certainly better.

The common mistake is reaching for multi-agent as a solution to the problem of long or complex prompts. If a single agent is struggling with a complex task, the instinct is to break it into multiple agents that each handle part of the problem. But this adds coordination complexity without necessarily fixing the underlying issue. A better prompt or a better context management strategy usually solves the problem without the multi-agent overhead.

Before designing a multi-agent system, ask three questions: Does the task have genuinely independent subtasks that benefit from parallelism? Are there distinct roles that require genuinely different capabilities or context? Can I afford the latency of agent-to-agent communication? If the answer to all three is yes, multi-agent makes sense. If any answer is no, reconsider.

Coordination patterns that actually work

The orchestrator-worker pattern is the most reliable multi-agent coordination pattern for practical automation. One orchestrator agent receives the high-level task, decomposes it, dispatches subtasks to worker agents, collects their outputs, and synthesises the final result. The orchestrator is responsible for the overall plan; workers are responsible for their specific tasks only.

The critical design principle is that workers should not need to communicate with each other. Agent-to-agent communication creates dependency chains where one agent waits on another, and failure in one propagates to all downstream agents. If workers do need to share information, have them return their outputs to the orchestrator and let the orchestrator relay relevant context — don't create direct dependencies between workers.

The orchestrator also handles replanning when workers fail. If a worker returns an error or a below-threshold quality result, the orchestrator should decide whether to retry with the same worker, try a different approach, or escalate. The orchestrator is the locus of system-level reasoning about the task as a whole; it should have the authority to change the plan when circumstances change.

Memory: the thing that makes or breaks multi-agent systems

Multi-agent systems need shared memory — a way for agents to read and write state that persists across the lifetime of a task. Without shared memory, each agent starts cold, unaware of what other agents have done or decided. With poor shared memory design, agents overwrite each other's state or read stale values.

The simplest approach that works reliably: a structured JSON state object that the orchestrator maintains and passes explicitly to each worker. The orchestrator reads the current state, decides what to pass to each worker based on what that worker needs to know, and updates the state with each worker's output. Workers don't read or write the state directly — they receive context as input and return structured output. The orchestrator manages all state transitions.

For longer-running tasks where workers need to build on previous work, persistent storage is necessary. A simple key-value store — Redis, or even a Postgres table — where each task gets a unique ID and agents read and write state by task ID works well. Include a schema version in the state object so you can evolve the state structure without breaking in-progress tasks.

Failure handling: assume everything will fail

Multi-agent systems fail in ways that single-agent systems don't. An individual agent might fail to complete its task, return an unexpected output format, exceed its context window, or hit an API rate limit. At the system level, these individual failures need to be caught, categorised, and handled without corrupting the overall task state.

Every agent call should be wrapped in error handling at the orchestration layer. If a worker fails, the orchestrator needs to know: was it a transient error (retryable) or a deterministic failure (needs a different approach)? A network timeout is retryable. An output that violates the expected schema after three retries is a signal that the worker needs different input or a different prompt. The orchestrator should categorise failures and respond accordingly, not just retry blindly.

Define a timeout for every agent call. An agent that doesn't return within the expected window is worse than an agent that fails — it blocks the orchestrator and delays all downstream work. In practice, most agent calls that take more than twice their average latency have encountered a problem. Set the timeout at three times average latency, and treat timeout as a failure that triggers the normal failure handling path.

“A multi-agent system's reliability is determined by its worst-case handling, not its happy-path performance.”

Testing multi-agent systems before they go to production

Testing multi-agent systems is harder than testing single-agent ones because the failure modes are combinatorial. A system with five agents where each can fail in three ways has fifteen possible single-point failures and many more combinations. You can't test everything, but you can test the scenarios that matter most.

Start with failure injection testing: manually cause each worker to fail and verify that the orchestrator handles it correctly. Does it retry? Does it escalate? Does it produce a coherent partial result? Does it log enough information to diagnose the failure? These tests should be automated so you can run them after every prompt change.

Also test the boundary conditions: what happens when a worker returns output that's technically valid but semantically wrong? What happens when two workers return conflicting information that the orchestrator has to resolve? These edge cases are harder to test systematically, but spending an hour working through them manually before deploying will surface the failure modes that cause production incidents.