Building Agent Networks

Why single-agent fails at scale

A single-agent loop — one model, one context window, one sequence of tool calls — is the right architecture for tasks that fit within a few dozen steps and stay within the context budget. Most interesting tasks do not fit this description. When a task is too long, too broad, or too error-prone for a single agent to complete reliably, you need a multi-agent architecture.

Context limits

A single-agent loop accumulates context with every step — tool calls, results, reasoning, and prior outputs all grow the context window. A 200K token context window sounds large until you realize that a 30-step task with moderate tool results can consume it entirely. The agent that starts with 200K tokens available has perhaps 10K tokens left by step 25, constraining its ability to reason about what it has learned. Multi-agent architectures reset the context window at task boundaries — each subagent starts fresh, with only the context it needs.

Task breadth

Some tasks require multiple specialized capabilities that are in tension when combined in a single agent. An agent that is simultaneously trying to search for information, write code, and manage a file system may do all three tasks less well than three specialized agents each focused on one. Specialization is a design principle: narrower context, narrower tools, and a clearer success criterion produces more reliable behavior than a generalist agent with everything in scope.

Error compounding

In a long agentic sequence, early mistakes propagate forward. The agent that made an incorrect assumption on step three will carry that assumption through steps four through thirty, and the final output will be wrong in ways that are hard to trace back to the root cause. Short, validated subagent tasks create natural checkpoints where outputs can be verified before downstream work depends on them. Errors compound less severely when the chain is shorter.

Orchestrator patterns

There are three primary orchestration patterns, each suited to a different task structure. Most real systems combine elements of more than one.

Planner-executor

A planner agent receives a high-level goal and produces a structured plan: a sequence of subtasks with defined inputs, outputs, and dependencies. The executor (which can be the same model or a different one) works through the plan, calling specialized subagents or tools for each subtask. The planner and executor have different system prompts optimized for different skills — the planner reasons about task decomposition; the executor focuses on reliable execution.

Planner-executor works well for tasks where the structure of the work is known in advance — code generation pipelines, document processing workflows, structured research tasks. The plan provides a shared artifact that both the executor and a monitoring layer can use to track progress and detect failures.

Fan-out

An orchestrator dispatches the same task, or partitions of a task, to multiple subagents running in parallel. The results are aggregated by the orchestrator after all subagents complete. Fan-out is the right pattern for tasks that can be decomposed into independent subtasks — analyzing multiple documents, searching multiple sources, generating multiple solution candidates for comparison.

Fan-out dramatically reduces wall-clock time for parallelizable tasks. A 10-document analysis that takes five minutes sequentially takes thirty seconds with a fan-out of ten parallel agents. The cost per task is the same; the latency is transformed.

Pipeline

A sequence of agents where each agent's output is the next agent's input. Stage one extracts, stage two transforms, stage three validates, stage four formats. Pipelines are the right pattern for tasks with clearly defined transformation stages where earlier stages filter or structure data for later stages.

Each pipeline stage should have a well-defined input contract and output contract. An agent that receives unstructured text and produces structured JSON is a reliable pipeline component. An agent whose output format varies based on input characteristics is a liability — it will break the next stage in the pipeline under edge cases.

Subagent interface design

A subagent is an agent called by an orchestrator to complete a scoped task. Subagent interface design determines how reliably the subagent completes its task and how easy it is to replace or update a subagent without affecting the rest of the system.

Scoped context

Give each subagent only the context it needs to complete its task. Do not pass the full conversation history, the full system state, or documents that are irrelevant to the subtask. Scoped context reduces distractions, keeps the token count low, and makes the subagent's behavior more predictable. An agent that knows less irrelevant information makes fewer irrelevant decisions.

Design the subagent's context explicitly: what does it need to know to do its job? The task description, the relevant data, the output format, and any constraints on how it should work. Nothing else. If you find yourself including everything "just in case," you have not scoped the task tightly enough.

Narrow tools

Limit each subagent to the tools it actually needs. A subagent that writes files should have a write-file tool, not a full filesystem API. A subagent that queries a database should have a single read-query tool, not a full database management interface. Narrow tools reduce the blast radius of mistakes — an agent that can only write specific file types cannot accidentally delete the whole file system.

Narrow tools also improve performance. An agent with five relevant tools uses them more accurately than an agent with fifty tools, most of which are not relevant to the current task. The tool selection problem becomes easier when the choice set is smaller.

Clear output contracts

Define exactly what the subagent should return: the format, the fields, the constraints. If the output will be parsed programmatically, require a structured format (JSON with a specific schema) and validate it before passing it downstream. If the output is natural language that will be read by the orchestrator, specify the expected structure (e.g., "return a bulleted list of findings followed by a recommended action").

Treat the output contract as an API contract. Changes to it require updates to downstream consumers. If you design it as an afterthought, you will spend significant time debugging integration failures between subagents.

Failure modes and recovery

Multi-agent systems fail in more ways than single-agent systems. Each agent adds a failure point; each handoff is a potential corruption or loss of information. Designing for failure is not optional — it is the core engineering problem of multi-agent architectures.

Retry strategies

Most transient failures — rate limits, temporary API errors, malformed outputs — resolve with a retry. But naive retry (try the exact same request again) is not always the right strategy. For an agent that produced malformed JSON, retrying with the same prompt may produce the same error. A better retry strategy includes the error in the context: "Your previous response was not valid JSON. The error was: {error}. Please retry, producing only valid JSON matching the schema." This gives the model the information it needs to correct itself.

Set explicit retry limits. An agent in an infinite retry loop will exhaust your API budget, your time budget, or both. Three retries is usually sufficient; if a task fails three times with corrective feedback, the task definition or the model is the problem, not the transient error.

Validation layers

Insert validation between pipeline stages. Before the orchestrator passes a subagent's output to the next stage, validate it against the expected schema or structure. A JSON output that does not parse, a structured extraction that is missing required fields, a classification result outside the expected label set — these should all be caught and handled before they corrupt downstream processing.

Validation can be programmatic (JSON schema validation, type checking) or model-based (a lightweight verification pass that checks for specific properties). Programmatic validation is cheaper and faster; use it for structural properties. Model-based validation is more flexible; use it for semantic properties like "does this extracted summary accurately represent the source document."

Circuit breakers

A circuit breaker stops retrying after a threshold of failures and returns an error to the caller rather than continuing to fail. This prevents a failing subagent from holding up the rest of the pipeline indefinitely and prevents runaway costs from a retry loop that is not converging.

Implement circuit breakers at the orchestrator level. Track failure counts per subagent, per time window. If a subagent exceeds the threshold, open the circuit: stop sending requests to it, surface a clear error to the orchestrator, and let the orchestrator decide whether to abort the task, route to an alternate agent, or degrade gracefully.

Observability

Multi-agent systems are opaque by default. Multiple agents, multiple tool calls, multiple handoffs, and all of it happening in sequence or in parallel — without deliberate instrumentation, debugging a production failure is nearly impossible. Observability is not optional infrastructure. Build it before you need it.

Tracing agent runs

Assign a trace ID to every top-level task and propagate it through all subagent calls. Every log line, every API call, every tool execution in the context of that task should carry the same trace ID. This makes it possible to reconstruct the full execution path of any task after the fact — what the orchestrator decided, what each subagent received and returned, what tools were called and with what arguments.

Use a structured tracing format — OpenTelemetry is the standard — and store traces in a system you can query. The ability to search for all traces that involved a specific subagent, or all traces where a specific tool returned an error, is what makes debugging tractable.

Logging tool calls

Log every tool call with the full input arguments and the full output. This is the most important single instrumentation point in an agentic system. Tool call logs tell you exactly what the agent tried to do and what it got back. When a task produces a wrong answer, the tool call logs show whether the error was in the inputs the agent chose (bad reasoning), the tool itself (implementation bug), or the downstream processing of the result (parsing error).

Tool call logs also produce your eval dataset. When you encounter a production failure, extract the failing trace, identify the problematic tool call, and add it to your eval suite. The production failure becomes a regression test.

Debugging misbehavior

When an agent produces unexpected behavior, the debugging process follows a standard sequence: find the trace, identify the step where the behavior diverged from expected, examine the full context at that step (what the agent was told, what tools it had, what its prior context contained), and determine whether the error was in the input context, the model's reasoning, or the tool execution.

Most agent misbehavior falls into one of three categories: underspecified instructions (the agent was not told what to do in this situation), context pollution (irrelevant or incorrect information in context led the agent astray), or tool failure (the tool returned wrong data and the agent reasoned correctly from wrong premises). The debugging process should distinguish between these — the fix for each is different.