Agent Orchestration
How it works
The orchestrator (also called supervisor, planner or manager) receives the task and dynamically decomposes it into subtasks which — unlike static parallelization — are not predefined but determined from the input. Subtasks are routed and delegated to worker agents; each worker is an LLM equipped with its own instructions, tools and scoped context. Agents communicate either by handing off control (a handoff carrying a destination and payload, e.g. the Command object in LangGraph) or through a shared state object or message history. Worker outputs return to the orchestrator (or are aggregated programmatically) and are synthesized into a final answer. State and memory are maintained through shared conversation history or an explicit state object, and stopping conditions (such as a maximum number of iterations) guard against runaway execution.
Problem solved
A single agent loses effectiveness when it (a) has too many tools and makes poor decisions about which to call, (b) accumulates context too complex to keep within the context window, and (c) must cover multiple specialization areas at once. Orchestration isolates context per agent and lets specialists handle narrowly scoped subtasks.
Components
The central agent that receives the task, dynamically breaks it into subtasks, delegates them to workers and combines their outputs. In Anthropic's orchestrator-workers workflow it both plans and synthesizes; in LangGraph's supervisor architecture it decides which agent is called next.
A specialized agent equipped with its own instructions, tools and scoped context that performs a focused subtask and returns results to the orchestrator. Called "sub-agents" in the OpenAI Agents SDK.
Official
A routing step that classifies an input and sends it to a specialized agent or follow-up task. In LangChain a router is typically stateless, requiring an LLM routing call per request.
Official
The mechanism that carries context across agents — a shared conversation history or an explicit graph state object (e.g. LangGraph graph state updated via Command.update).
Official
The protocol transferring control plus payload between agents — a handoff carrying a destination and state update (e.g. LangGraph Command, or a transfer_to_XXX function in the OpenAI cookbook pattern).
Official
Implementation
Errors produced by one agent propagate and amplify across the chain of agents. Anthropic: "the autonomous nature of agents means higher costs, and the potential for compounding errors."
Multiple LLM calls per task multiply tokens and wall-clock time. Anthropic: agentic systems "trade latency and cost for better task performance."
Each agent sees only part of the state; poor context partitioning degrades quality. LangChain: "at the center of multi-agent design is context engineering — deciding what information each agent sees."
Agents can route to themselves or loop without a stop condition. Anthropic recommends stopping conditions "such as a maximum number of iterations to maintain control."
The supervisor/router adds extra LLM calls, and poor routing decisions harm the whole system. Routers are typically stateless, requiring an LLM routing call per request.
Model-driven routing makes behavior hard to reproduce, and frameworks can obscure the underlying prompts. Anthropic: frameworks "create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug."
Using a multi-agent system when a single LLM call would suffice adds cost and fragility. Anthropic: "find the simplest solution possible … This might mean not building agentic systems at all."
Evolution
Yao et al. introduce the reason-and-act loop that underpins single-agent tool-using behavior which orchestration later composes into multi-agent systems.
Open-source projects AutoGPT (repo created 2023-03-16) and BabyAGI (2023-04-03) popularize autonomous task-decomposition loops, foreshadowing orchestrated multi-agent setups.
Wu et al. publish AutoGen (arXiv:2308.08155, submitted 16 Aug 2023), a framework of customizable conversable agents that converse to solve tasks — a canonical academic reference for multi-agent conversation.
LangChain launches LangGraph (17 January 2024), providing explicit supervisor, network and hierarchical multi-agent architectures with stateful control flow.
OpenAI publishes Swarm (GitHub repo created 2024-02-22), an educational framework built on two primitives — Agents and handoffs — exploring ergonomic multi-agent orchestration.
Anthropic publishes "Building Effective Agents" (19 Dec 2024), the most-cited articulation of the orchestrator-workers pattern and the workflow-vs-agent distinction.
OpenAI releases the Agents SDK (GitHub repo created 2025-03-11; PyPI openai-agents v0.0.1 on 2025-03-04), formalizing handoffs and agents-as-tools for production multi-agent workflows.
Hyperparameters (configurable axes)
The topology used to coordinate agents: supervisor, network, hierarchical, or a custom workflow. Determines how control flows between agents.
How agents transfer control and context: handoffs (transfer the active conversation) versus agents-as-tools (manager keeps control and calls specialists as tools).
The limit (e.g. maximum number of iterations) that bounds autonomous loops and keeps the system under control. Anthropic recommends explicit stopping conditions.
Deciding what information each agent sees (context engineering). Poor partitioning degrades quality; strong isolation keeps each agent focused.
Compute bottleneck
Each orchestration step issues one or more LLM calls; sequential dependencies between orchestrator and workers accumulate wall-clock latency and token cost. Anthropic notes that agentic systems "trade latency and cost for better task performance".
Execution paradigm
Which agent is activated depends on the input and the orchestrator's decision — hence conditional mode and input_dependent activation.
A central orchestrator (or router) decides, based on the input, which worker agent to invoke next and with what payload. Implemented via handoffs (transfer of the active conversation) or agents-as-tools (the manager calls specialists as tool calls). Only the agents relevant to the current subtask are activated.
Parallelism
Parallelism is pattern-dependent: sectioning and voting (independent subtasks) run concurrently, while chains of dependent steps proceed sequentially.
Hardware requirements
Orchestration is an application/runtime-level paradigm realized through LLM API calls and control logic; it is independent of any specific accelerator and runs wherever the underlying models are served.