Major

Multi-Agent Orchestration: Patterns and What They Cost

Every ranking guide on multi-agent orchestration explains the patterns and stops there. None of them asks what it costs to have a model re-derive the same routing decision on every single run. Here are the patterns, and the point where coordination should stop being a reasoning p

Rahul RamakrishnanUpdated
Conductor leading an orchestra in a concert hall, a picture of coordinating many specialists

Multi-agent orchestration is the coordination of several specialized AI agents across one multi-step workflow: which agent runs, in what order, with which tools, and with what authority to act. Four patterns cover most designs (sequential, concurrent, handoff, and hierarchical), and each one answers the same question differently. How much of the coordination should a model decide at runtime?

Most guides stop at the patterns. The part that decides whether a system survives production is what happens on run one thousand. A model that re-derives the same routing decision on every run charges you for it every time and can answer differently on Tuesday than it did on Monday. Major's answer is to let the agent reason once, build the routing and state into a deterministic app, and run the app from then on. The model keeps the judgment calls and stops carrying the settled ones.

What multi-agent orchestration is, and what it is not

It is narrower than AI orchestration in general, which covers models, retrieval, tools, and prompts in any arrangement. Multi-agent orchestration is about agents as the coordinated units.

It also differs from workflow automation on exactly one axis: who decides the next step. In workflow automation someone wrote the routing rule in advance and the runtime follows it. In multi-agent orchestration a model can make that call at runtime. Nearly every production system is a mix, so the engineering work is deciding which steps sit on which side.

Name the two layers. The agent layer reasons, plans, and picks. The app layer is deterministic code where work executes and state lives. Every pattern below is a different split between them.

Why single agents hit a ceiling

Context fills up, tool inventories grow past the point where the model chooses well, and one prompt ends up carrying four jobs. More agents looks like the fix.

The primary documentation is more cautious than the discourse. Microsoft's Azure Architecture Center puts agent design on a complexity ladder, from a direct model call to a single agent with tools to multiagent orchestration, and tells readers to "use the lowest level of complexity that reliably meets your requirements." Multi-agent systems are not better by default. Single agents are running out of room, and teams are choosing how to extend them.

The four coordination patterns

Sequential. Agents run in a fixed linear order, each consuming the last one's output. It fits draft, review, polish shapes. Its failure mode is error propagation: a weak stage-one output feeds stage two, and by stage four you are scoring a contract built on the wrong template.

Concurrent. Several agents work the same input at once and an aggregator merges the results. Fan-out and fan-in, scatter-gather, and map-reduce are the same shape. It breaks on aggregation and shared state. Reconciling five disagreeing outputs is itself work, and Azure warns against the pattern when agents cannot reliably coordinate changes to shared state.

Handoff. Each agent decides whether to finish the task or pass it to a better specialist. Triage and routing are the same pattern under other names. The failure mode is bouncing, where agents loop a ticket between them because each decides someone else should own it. Azure's own guidance is the sentence this article leans on: avoid handoff orchestration when "task routing is deterministic and rule-based." In that case you are paying a model to run a lookup.

Hierarchical. A supervisor delegates to workers and synthesizes what returns. It is the pattern most people picture. Two constraints matter. Depth is not free: Anthropic's managed agents documentation states that "the coordinator can only delegate to one level of agents," and rejects rosters that nest. And delegation crosses permission boundaries. Copilot Studio's guidance notes a connected agent "might have access to things the parent agent doesn't," which means a parent barred from deleting records can call a child that is allowed to. That is privilege escalation drawn as an architecture diagram, and it argues for control at the point of action rather than at the top of the tree.

Where coordination state lives

Take a support triage flow. A supervisor classifies a ticket and routes it to a billing agent, a technical agent, or an escalation path. It is the most common multi-agent system in production.

In the default design the ticket state lives in context windows. Three things break. A mid-run failure leaves nothing to resume from. There is no continuity between runs, so ticket 400 from the same customer arrives as a stranger. And every run pays to refill context the last run already assembled.

Dataiku's agent orchestration guide splits memory tiers from an "immutable audit trail" of actions and tool calls and treats them as two concerns. They are one. A durable coordination store that records which agent received which ticket, under whose credentials, and what it did serves as both memory and audit log.

What orchestration costs on run one thousand

Coordination decided by a model is a recurring bill. The supervisor classifies ticket one, then classifies ticket one thousand, spending tokens both times on a decision whose rules stopped moving around ticket two hundred.

Nobody in the ranking set quantifies this, and I will not manufacture a number. The OpenAI Agents SDK comes closest, separating orchestrating via LLM from orchestrating via code and stating that "orchestrating via code makes tasks more deterministic and predictable." The argument here is structural. Model-decided coordination costs in proportion to usage. Code-decided coordination costs once to establish, then flattens. Front-loaded, then flat, against climbing with volume.

The tradeoff is real. If your routing genuinely changes week to week, a model deciding it is the right call and hard-coding the rule is the wrong one.

When coordination should stop being a reasoning problem

Ask three questions of each step, answerable from your own logs.

  1. Over the last few hundred runs, how often did this step reach a different decision for the same shape of input? Close to zero means the step is a lookup dressed as reasoning.
  2. Can you write the rule in a sentence a colleague would sign off on? Then it belongs in code.
  3. Does the decision need information that only exists at runtime and cannot be enumerated ahead of time? Then keep it with the model.

Build the triage orchestration on Major

Here is the triage flow above, built so the settled coordination leaves the model.

  • Connectors. Zendesk for inbound tickets and Slack for escalations, attached with scoped credentials so each agent acts only through what it was granted.
  • The app that holds state. A triage app on Major's managed database with tables for tickets, routing rules, and handoff records. It replaces the context window as memory.
  • The routing rules. The agent works out the routing table once, builds it into the app, and the app applies it as code on every later ticket.
  • The agent and its judgment. The agent handles only the ambiguous ticket that matches no rule, and is the source of the next rule to move into the app.
  • The trigger. A workflow fires on new Zendesk tickets and runs the app first, calling the agent only on a miss.
  • Governance. Role-based access on the app, scoped credentials per connector, and an audit log of every routing decision and side effect, so you can answer what happened in March without replaying a transcript.

Start building this on Major at app.major.build.

Common misconceptions

More agents does not mean more capability. Copilot Studio warns that two subagents on the same knowledge source produce one useful answer and one redundant search.

A supervisor is not a scheduler. A scheduler is deterministic and costs nothing per decision. A supervisor is an inference call with an opinion.

A long context window is not state. It is a bigger buffer that still empties. State is a record that outlives the run.

What we are building at Major in response

Coordination that has stopped changing should stop being a model call. Major is the enterprise platform where agents build the software they run on, so the triage routing table becomes code, and the ticket state and handoff record live in a managed database instead of a session that ends. The model still does the hard part. It handles judgment and notices the next piece of coordination that has settled. Reason once. Run forever.

This argument does not cover agent evaluation, a harder problem that deserves its own piece. It also assumes your settled steps are in fact settled. If they are not, keep them with the model.

Start building on Major

Build a triage or routing app where agents reason once, the app runs the settled coordination, and every handoff is logged under scoped credentials.

Related articles

Frequently asked questions

What is multi-agent orchestration?
Multi-agent orchestration is the coordination of several specialized AI agents across a single multi-step workflow, deciding which agent runs, in what order, and with what authority to act. Four pattern families cover most designs: sequential pipelines, concurrent fan-out, handoff routing between specialists, and a hierarchical supervisor delegating to workers.
What is the difference between multi-agent orchestration and workflow automation?
The difference is who decides the next step. Workflow automation follows a routing rule someone wrote in advance. Multi-agent orchestration lets a model make that call at runtime, reading context and choosing where work goes. Most production systems combine both, and the real engineering decision is which steps sit on which side of that line.
When should you use multiple agents instead of one?
Use multiple agents when subtasks are genuinely parallelisable, when agents need different tool permissions or security boundaries, or when contexts must stay isolated to keep output quality up. Microsoft's guidance says to use the lowest level of complexity that reliably meets your requirements, so start with one agent and split only at a real boundary.
Where does state live in a multi-agent system?
By default it lives in context windows, held by each agent for the life of the session. That breaks in three ways: a mid-run failure leaves nothing to resume from, there is no continuity between runs, and every run pays to refill context the last run already assembled. A durable store outside the agents fixes all three and records who did what.
Does multi-agent orchestration increase cost?
Usually yes, through two mechanisms. Coordination itself consumes inference, since a supervisor or routing agent is a model call. And that call repeats on every run, re-deriving decisions whose rules stopped changing long ago. Move settled coordination into code and cost is front-loaded then flat. Leave it with the model and cost climbs with volume.