Multi-Agent Systems Need Application Architecture Before Orchestration Frameworks
TOGAF 10 provides the Enterprise Architecture method, including the Application Architecture discipline within Phase C. The agent decomposition and orchestration decision framework presented in this article is my applied methodology for using that discipline in AI-native systems; it is not a prescribed TOGAF technique.
Why Enterprises Are Building Agent Orchestration Before They Design It
Every enterprise AI conversation in 2026 eventually arrives at the same fork. A team has a business problem. Somewhere in the discovery process, someone proposes multi-agent orchestration as the solution. A framework gets selected. Agents get named. Handoffs get wired. Only much later does anyone ask whether the problem needed more than one agent at all.
This sequence is backwards. Orchestration pattern is an architecture decision. Framework selection is a tooling decision. Enterprises that let tooling decide architecture end up with systems that are expensive to run, difficult to debug, and nearly impossible to govern.
This article argues for a different discipline. Before any framework gets chosen, the application architecture should already answer three questions. Does this problem need one agent or many. If it needs many, do they run in sequence or in parallel. What does failure look like at each boundary. Only after those answers exist should a framework enter the conversation.
The Framework-First Instinct, and Why It Is Seductive
Framework-first thinking is not a mistake made by careless teams. It is the natural output of how agentic AI tooling has been marketed and adopted.
Vendors ship reference architectures. Those reference architectures default to multi-agent patterns because multi-agent demos are visually impressive in a way single-agent pipelines are not. A supervisor agent delegating to three specialist agents looks like sophisticated engineering. A single agent calling a well-designed tool chain looks unremarkable, even when it solves the same problem more reliably.
Engineering teams under delivery pressure gravitate toward whichever pattern the documentation shows first. The framework's default topology becomes the project's architecture by inertia rather than by decision. Nobody signs off on this. It simply happens, because no one was asked to sign off on anything.
The cost of this inertia rarely appears in the pilot. It appears in production, six months later, when the finance team asks why a document summarization workflow requires four agents, three retries, and two thousand tokens of inter-agent chatter to do what a single well-prompted call could do in one pass.
From Business Need to Application Architecture
The decision does not begin in Phase C. The need for an agentic solution should first be grounded in the Architecture Vision, stakeholder concerns, business requirements, and the scope established for the architecture engagement. Phase C then translates those requirements into the logical Application Architecture needed to support the target state. Requirements Management remains continuous throughout the ADM, feeding requirements into the relevant architecture work and receiving changes back from it.
Orchestration Pattern Is an Architecture Decision, Not a Tooling Decision
Enterprise architecture has always separated two questions that agentic AI projects now routinely conflate. The first question is what the system needs to do and how its parts should be structured to do it. The second question is what technology implements that structure.
Application architecture answers the first question. It defines the logical shape of a solution before any product, library, or framework enters the picture. This is the same architectural discipline that TOGAF applies to application architectures more broadly, and it provides a useful foundation for architecting agentic systems. The specific decision criteria used for agent decomposition and orchestration, however, are an AI-specific application of that discipline, not prescriptions defined by TOGAF itself.
TOGAF's Architecture Development Method places this work in Phase C, Information Systems Architectures, which itself splits into two coordinated tracks: Data Architecture and Application Architecture. In an agentic system, the Application Architecture perspective can therefore be used to define the logical responsibilities of agents, tools, orchestration components, interaction patterns, and supporting application services, while the Data Architecture perspective addresses the information structures and data flows those components depend upon. The precise modeling approach should be tailored to the architecture scope and stakeholder concerns.
TOGAF divides Phase C into two parts covering the development of the Data and Application Architectures, with a brief introductory chapter covering both domains and then a separate chapter for each. The Application Architecture track exists specifically to define the major kinds of application system necessary to process the data and support the business, described not as computer systems but as logical groups of capabilities that manage data and support business functions.
That last distinction matters enormously for agentic systems. TOGAF is explicit that Phase C application work is not concerned with application systems design in the implementation sense. It is concerned with what kinds of application components are needed, and what those components must do. An orchestration pattern, such as a single-agent or multi-agent structure and the dependency relationships between agent activities, can be treated as an Application Architecture decision within this article's AI-specific application of TOGAF. The architecture should establish the required logical application components, responsibilities, interactions, and dependencies before implementation technology is selected. It does not belong in a framework's getting-started guide.
Confirming the standard further, the objectives of the Application Architecture part of Phase C are to develop the Target Application Architecture that enables the Business Architecture and Architecture Vision, addressing the Statement of Architecture Work and stakeholder concerns, and to identify candidate roadmap components based on gaps between baseline and target architectures. Orchestration pattern selection is a direct expression of that target architecture. It cannot be delegated to a framework default without abandoning the discipline TOGAF was built to enforce.
Application Architecture Does Not Exist in Isolation
Agent orchestration cannot be designed independently of the information architecture that supplies its context.
A multi-agent design may introduce additional requirements for state, memory, context propagation, retrieval, shared knowledge, provenance, and synchronization. These are not merely framework concerns. They create architectural dependencies between Application Architecture and Data Architecture.
A useful architectural separation is therefore:
Business requirements → Application Architecture → orchestration structure
alongside:
Information requirements → Data Architecture → context, knowledge, state, and data flows
The two must converge in the Target Architecture.
From Baseline to Target Architecture
In an enterprise setting, the orchestration decision should also be considered against the Baseline and Target Application Architectures. If the current platform already provides deterministic workflow orchestration, for example, introducing an agentic orchestration layer represents an architectural change that should be justified through the gap between the baseline and target states.
The resulting gap analysis can identify the application components, capabilities, integration changes, data requirements, and implementation work required to move from the Baseline Architecture to the Target Architecture.
This keeps the orchestration decision connected to the broader architecture roadmap rather than treating it as an isolated technology decision.
The Core Thesis, Stated Plainly
An orchestration framework is an implementation choice. It should be selected after the application architecture has already determined the shape of the solution, never before.
A single agent with well-designed tools and a disciplined prompt can outperform a five-agent swarm on cost, latency, and reliability, for a large share of enterprise use cases. Multi-agent decomposition is a legitimate and sometimes necessary pattern. It becomes a liability the moment it is chosen because a framework made it the path of least resistance, rather than because the problem's structure demanded it.
Enterprise Architects, solution architects, engineering leaders, and AI leaders can play an important role in intercepting this failure mode by ensuring that implementation choices remain traceable to architectural requirements. Engineering teams optimize for shipping something that works in a demo. Architecture Governance provides the broader governance framework for architecture development and evolution, while Phase G, Implementation Governance, provides architectural oversight during implementation and helps ensure that delivered solutions conform to the approved architecture.
A Decision Framework for Orchestration Pattern
Application architecture decisions should be made against explicit criteria, not intuition. The following framework gives Enterprise Architects a structured way to reach a defensible answer before any framework conversation begins.
Step Zero: Establish the Architectural Requirements
Before deciding whether an agentic workflow should be decomposed, establish the requirements that the target application architecture must satisfy. These include business outcomes, functional requirements, performance and latency expectations, availability, security and privacy constraints, auditability, human involvement, scalability, cost constraints, and failure-tolerance requirements.
The orchestration pattern should emerge from these requirements rather than become a requirement in itself.
Step One: Determine Whether the Problem Is Decomposable
A problem is decomposable when it contains genuinely independent subtasks, each requiring a distinct skill, tool boundary, or context window that would otherwise be diluted by combination.
Document summarization, single-turn question answering, and most retrieval-augmented generation tasks are not meaningfully decomposable. They are single coherent reasoning tasks wearing the costume of complexity. Complex research synthesis, multi-system remediation, and workflows spanning genuinely separate domains of expertise often are decomposable.
If the problem does not require meaningful decomposition, there may be no architectural justification for multiple agents. The solution may be a single agent, a conventional application workflow, deterministic automation, or another pattern altogether. The important architectural conclusion is that multi-agent orchestration should not be introduced without a demonstrated requirement for it.
Step Two: If Decomposable, Determine Dependency Structure
Subtasks that must complete in order, where each step's output shapes the next step's input, describe a sequential pattern. Subtasks that are mutually independent, where none requires the output of another to begin, describe a candidate for parallel execution.
Most enterprise workflows contain a mixture. A claims processing workflow might parallelize document extraction across multiple document types, then sequentially route the combined extraction through a rules engine and a decision agent. Mapping this dependency graph honestly, subtask by subtask, is the actual work of Phase C application architecture for agentic systems.
Step Three: Weigh the Cost of Coordination Against the Cost of Combination
Every agent boundary introduces coordination cost. Context has to be serialized, passed, and reinterpreted at each handoff. Every handoff is a place where meaning can degrade, latency can accumulate, and failure can hide.
Every combination into a single agent introduces a different cost. A single context window carrying too many responsibilities becomes harder to prompt reliably, harder to evaluate, and harder to debug when it fails in one of several possible ways at once.
The architecture decision is a genuine trade-off, not a default. Enterprise Architects should require teams to state this trade-off explicitly, in writing, before implementation begins. Where the justification for coordination is weak, the simpler architecture should be preferred until additional requirements demonstrate the need for decomposition.
Step Four: Confirm Failure Isolation Requirements
Some enterprise workflows require that a failure in one part of the system not corrupt or block the rest. A multi-agent architecture with isolated agents can contain failure at a boundary, retry that boundary alone, and preserve everything upstream.
A single agent carrying the entire workflow in one context has no such isolation. A failure partway through can be harder to localize, and a retry often means repeating the entire task from the start.
Where regulatory audit trails, partial-completion tolerance, or blast-radius containment matter, this criterion can justify multi-agent decomposition even for tasks that are only moderately decomposable. This is precisely the kind of non-functional requirement that TOGAF's Requirements Management discipline exists to surface early, rather than letting it emerge as an incident review finding after launch.
Step Five: Compare Candidate Architectures
The decision should not be framed as "single agent versus multi-agent" alone. Candidate architectures may include deterministic workflow automation, a single agent with tools, sequential multi-agent orchestration, parallel multi-agent orchestration, or a hybrid architecture.
The selected architecture should be the one that satisfies the stated requirements with acceptable complexity, risk, cost, and operational characteristics.
Step Six: Make the Orchestration Decision Traceable
A defensible architecture decision should be traceable from business need to implementation:
Business Driver → Business Requirement → Architectural Requirement → Application Architecture Decision → Orchestration Pattern → Technology Implementation
This prevents the framework from becoming the starting point of the architecture. The framework becomes the implementation mechanism for an already-established architectural decision.
For example, a business requirement to process claims within a defined latency target may produce an architectural requirement for bounded processing time. The Application Architecture may then identify independently executable processing activities, leading to a parallel orchestration pattern. Only after that architectural decision has been established should an orchestration framework be selected to implement it.
Failure Modes of Framework-Default Orchestration
Enterprises that skip application architecture and let framework defaults dictate orchestration pattern tend to encounter the same failures, repeatedly, across otherwise unrelated projects.
Unnecessary handoff latency accumulates silently. Each additional agent in a chain adds a round trip. A workflow that could resolve in one model call instead resolves in four or five, and nobody on the team can point to a decomposability requirement that justified the additional hops.
Context loss between agents degrades output quality. Information that was clear in an upstream agent's full context becomes ambiguous once summarized and passed to a downstream agent. This is not a framework bug. It is the structural cost of decomposition, paid whether or not decomposition was necessary.
Parallel execution can introduce race conditions and reconciliation complexity when concurrent agents share mutable state or produce outputs that must be merged. Without explicit synchronization and aggregation semantics, identical inputs can produce different intermediate states or reconciliation outcomes across runs.
Cost multiplies without a corresponding quality gain. Multi-agent patterns often mean multiple full model invocations where one would have sufficed. Finance teams eventually notice, and the resulting cost review tends to land on the architecture team's desk, not the framework vendor's.
Observability collapses across agent boundaries. Tracing a failure through five agents, three tool calls, and two retries requires instrumentation most teams did not build because the framework's own dashboards created a false sense that observability was already handled.
Each of these failure modes traces back to the same root cause. The orchestration pattern was inherited from a framework default rather than derived from the application architecture the workflow actually required.
When Multi-Agent Decomposition Is Architecturally Justified
None of the above should be read as an argument against multi-agent systems. It is an argument against choosing them by default. There are workflow shapes where multi-agent decomposition is not merely acceptable but provides a demonstrable architectural benefit.
Heterogeneous tool and skill boundaries justify separate agents when each subtask requires a genuinely different toolset, permission scope, or domain vocabulary that would otherwise bloat a single agent's context and increase the chance of tool misuse.
Adversarial and critique loops benefit from a structural separation between a generating agent and a reviewing agent, because a single agent reviewing its own output tends to rubber-stamp its own reasoning rather than genuinely challenge it.
Independently parallelizable subtasks with no shared state are close to a best case for multi-agent architecture, since the coordination cost is low, the latency benefit is real, and failure in one branch does not threaten the others.
Long-running, stateful workflows spanning multiple systems of record often benefit from persistent specialist agents that own a bounded piece of enterprise state over time, rather than a single agent attempting to hold the entire workflow's state in one context indefinitely.
In each of these cases, the decision framework above should produce a clear, defensible answer in favor of multi-agent decomposition. The point is not that multi-agent is wrong. The point is that the answer should come from analysis, not from whichever pattern a framework's quickstart guide happened to demonstrate first.
Embedding This Discipline in Enterprise Governance
A decision framework only changes outcomes if it is enforced somewhere in the delivery lifecycle. Three governance mechanisms make that enforcement durable.
Require an application architecture artifact before framework selection. As an applied governance practice, an agentic AI project entering delivery should produce a lightweight Application Architecture decision record before framework-specific implementation begins. The record should state the selected orchestration pattern, the architectural requirements behind it, the alternatives considered, and the criteria used to justify the decision.
Route the artifact through existing architecture governance. Enterprises that already have Architecture Governance mechanisms should incorporate agentic AI application architecture into those mechanisms, using governance structures and review criteria appropriate to the organization's architecture framework. Where an Architecture Board or equivalent governance body exists, it can provide the appropriate forum for reviewing significant orchestration decisions and architecture exceptions. Agentic AI does not deserve a governance exception simply because it is new.
Track orchestration pattern as a first-class field in the AI system inventory. Whatever registry an enterprise uses to track its AI systems for risk, cost, and compliance purposes should record orchestration pattern alongside model, data sources, and business owner. This turns an architecture decision into an auditable, queryable fact rather than an implementation detail buried in a repository.
Closing: Architecture First, Tooling Second
The framework you choose for multi-agent orchestration will change. LangGraph, CrewAI, AutoGen, Bedrock Agents, and whatever follows them will each have their season of dominance, their migration path, and eventually their successor. The application architecture underneath a well-built agentic system, by contrast, tends to outlive several generations of tooling, provided it was designed deliberately in the first place.
The discipline this article argues for is grounded in a broader Enterprise Architecture principle: establish the required architecture before allowing implementation technology to determine the solution shape. TOGAF provides that architectural discipline through its ADM, architecture domains, requirements management, governance, and implementation oversight. The specific decision framework presented here for agent decomposition and orchestration is an AI-specific application of those principles, not a prescriptive TOGAF technique.
The practical sequence is therefore:
Business Need → Requirements → Application Architecture → Orchestration Decision → Technology Selection → Implementation Governance
Choose the architecture first. Let the framework follow.
✍️ About the Author
Sanjoy Kumar Malik — Principal AI Architect, Enterprise AI Strategist, and Senior Engineering & Technology Leader with 20+ years of corporate IT experience and a broader 27+ year professional journey, spanning Enterprise Architecture, software architecture, cloud-native systems, engineering leadership, and AI architecture. He is a TOGAF 10 Certified Enterprise Architecture Practitioner and AWS Certified Solutions Architect – Professional.
Sanjoy focuses on translating business strategy and AI opportunity into coherent enterprise architecture and scalable engineering execution. He works at the intersection of business, technology, architecture, and AI, helping organizations establish the architectural foundations, technology capabilities, and engineering systems required to turn AI initiatives into production-grade, scalable, governed, and economically sustainable enterprise capabilities.
He is the creator of The 28-Category AI Architecture Decision Framework (28-CAADF), a systematic approach to making AI architecture decisions in an era where intelligence itself is becoming an architectural capability.