Agent Orchestration: One Agent Calling Many
Summary of the video “After This Video, You'll Actually Understand Agent Orchestration” by Burke Holland.
Agent orchestration lets a single AI agent delegate work to specialized sub-agents with different models, creating a virtual dev team. Using tools like Copilot CLI and VS Code custom agents, you can have an orchestrator (Sonnet) coordinate a planner (GPT-4o), coder (GPT-4o Codex), and designer (Gemini 3 Pro) — each with isolated context windows — to build complex projects efficiently.
What Is Agent Orchestration
From Manual to Automated Agent Management
Today, you manually run multiple agents in parallel (local, background, cloud) and coordinate them yourself. Agent orchestration automates this: one primary agent calls and coordinates specialized sub-agents, eliminating manual switching between models and tools.
Sub-Agents: Isolated, Specialized, Parallel
Sub-agents are specialized agents called by a main orchestrator. Each has its own model (GPT-4o, Gemini, etc.), its own isolated context window, and can run in parallel with others. Once a sub-agent finishes, its context is discarded and only the result returns to the orchestrator, keeping the main context window clean.
Building a Virtual Dev Team
Instead of one agent doing everything, you can architect a team structure: a team lead (orchestrator) assigns tasks to architects, coders, designers, planners, and PMs (all sub-agents). Each specialist uses the best model for their domain, working in parallel to solve complex problems faster.
How It Works in Practice
Copilot CLI: Simple Natural Language Delegation
In Copilot CLI, you can write a single prompt asking one model to review code and then run findings by GPT-4o and Gemini in parallel. The CLI automatically detects model names and delegates work without explicit tool configuration — it just works.
VS Code Custom Agents: Structured Framework
VS Code allows you to define custom agents with specific tools, prompts, and model assignments. You configure each agent's role, tools (like MCP servers for documentation), and the exact model it uses via a model line in the configuration.
The Orchestrator Role: PM, Not Coder
The orchestrator's sole job is to break down requests, delegate to specialists, and coordinate results — it never implements work itself. This requires explicit rules in the prompt to prevent the orchestrator from micromanaging sub-agents or writing code.
Model Selection by Specialty
Orchestrator: Claude Sonnet 4.5
Sonnet 4.5 is highly agentic and eager to delegate work, making it ideal for orchestration. It excels at breaking down tasks and coordinating but is not recommended for code generation.
Planner: GPT-4o
GPT-4o is used for strategic planning and breaking complex requests into actionable steps. It creates high-level plans that guide the entire orchestration workflow.
Coder: GPT-4o Codex
GPT-4o Codex specializes in code generation and is significantly better at writing production-quality code than other models. It's paired with MCP servers (like Context 7) to access documentation.
Designer: Gemini 3 Pro
Gemini 3 Pro produces superior UI/UX and design outputs compared to other models. It's given creative autonomy and minimal guardrails to focus purely on design quality.
Real-World Example: Building a Web App
The Workflow: Plan → Design → Code → Coordinate
The orchestrator receives a request to build a web experience for an iOS app. It calls the planner (which creates a detailed plan), then the designer (which creates a design system with CSS), then the coder (which builds the full application using the plan and design). The orchestrator coordinates handoffs between each stage.
Context Window Efficiency: 2,770 Lines, 10.8K Tokens
The orchestrator generated 2,770 lines of code while using only 10.8K of its context window. This is possible because sub-agents have isolated context windows — once they complete, their context is discarded and doesn't pollute the main orchestrator's window.
Parallel Sub-Agent Execution
Sub-agents can run in parallel. Instead of one coder agent doing all work sequentially, you could split tasks into discrete chunks and spawn five coder agents simultaneously, dramatically reducing total execution time.
Preventing Orchestrator Micromanagement
Models naturally want to tell sub-agents exactly what to do. The framework includes explicit rules like 'Question everything you're told' and 'Make your own decisions' to give sub-agents autonomy and prevent the orchestrator from over-specifying implementation details.
Practical Improvements & Alternatives
Optimization: Save Plans as Documents
When the planner creates a plan, it should save it as a document and always pass that document to the coder. This ensures the coder has full context rather than a high-level summary, improving code quality and alignment.
Optimization: Slice Work into Parallel Tasks
Instead of one coder agent handling all work, break it into discrete chunks and assign each to a separate coder agent running in parallel. This leverages the sub-agent parallelization capability for faster delivery.
Other Orchestration Frameworks
Beyond the lightweight framework shown here, more complex orchestration frameworks exist: AutoGen, Crew AI, and others. These offer advanced features but require more setup; start simple and graduate to complexity as needed.
Start Simple, Scale Gradually
You don't need 100 agents checking each other's code from day one. Begin with one orchestrator delegating to 3–5 specialized sub-agents. As you refine prompts and workflows, add complexity incrementally.
Notable quotes
You are the orchestrator. You are running all of these different agents. — Burke Holland
Sub-agents can be called by the main agent but sub-agents can have whatever model you want them to have. — Burke Holland
That's the magic of sub-agents. They have an isolated context window. They only use what's theirs. — Burke Holland
Action items
- Install the ultralight orchestration framework from the link below the video (Orchestrator, Planner, Coder, Designer agents for VS Code)
- Start with a simple orchestration setup: one orchestrator + 3–4 specialized sub-agents rather than building a complex multi-agent system
- Assign models strategically: Sonnet for orchestration, GPT-4o for planning, GPT-4o Codex for coding, Gemini 3 Pro for design
- Add explicit rules to your orchestrator prompt to prevent micromanagement of sub-agents (e.g., 'Don't tell sub-agents how to do their work')
- Test parallel sub-agent execution by splitting work into discrete chunks and assigning each to a separate agent
- Modify the planner to save detailed plans as documents and pass them to the coder to improve code quality