Skip to content

02 - Multi-Agent Systems ArchitectureΒΆ

How specialized AI agents work together to solve complex tasks through clear responsibilities, collaboration, and coordination.

IntroductionΒΆ

An individual AI agent can reason about a goal, use tools, observe results, and decide what to do next. That is often sufficient for a focused task.

But enterprise problems rarely stay within one capability. Investigating a payment incident, for example, may involve application logs, infrastructure metrics, deployment history, and downstream service behaviour. One agent can attempt all of these activities, but its instructions and context can become difficult to manage as responsibilities grow.

A multi-agent system distributes the work across agents with defined roles. Each agent focuses on a particular responsibility, and the system coordinates their contributions towards a shared objective.

The important design question is not how many agents we can create. It is where separation of responsibilities makes the system easier to reason about, operate, and improve.

1. Multi-Agent System FundamentalsΒΆ

A multi-agent system (MAS) consists of multiple agents that interact with one another and their environment to accomplish individual or shared objectives.

In an LLM-based system, an agent typically combines a model with instructions, context, tools, and a defined task. Agents may use the same underlying model or different models, depending on their responsibilities.

A basic multi-agent system has four elements:

  • Agents: perform reasoning and task-specific actions.
  • Environment: provides the data, tools, services, and operating conditions.
  • Communication: allows agents to exchange requests, messages, and results.
  • Coordination: manages task assignment, execution order, handoffs, and completion.
flowchart TD
    G["Shared Objective"] --> C["Coordination"]
    C --> A["Agent A"]
    C --> B["Agent B"]
    C --> D["Agent C"]
    A <--> E["Environment<br/>Data, Tools, Services"]
    B <--> E
    D <--> E
    A --> C
    B --> C
    D --> C
    C --> R["Combined Outcome"]

The coordinator does not have to be another LLM agent. It can be deterministic application logic, an agent, or a combination of both.

A multi-agent system is also not necessarily a distributed system. Several agents may run within one application process, while a distributed deployment may place them across separate services or machines. These are different architectural decisions.

2. Agent Roles and ResponsibilitiesΒΆ

Agents work better together when their responsibilities are explicit. A common design uses specialist agents to perform focused tasks and a coordinator to manage the overall objective.

Specialist agents handle a bounded area of work. For example:

  • A retrieval agent searches approved enterprise documents.
  • An analysis agent interprets retrieved evidence.
  • A review agent checks completeness and consistency.
  • A reporting agent prepares the final output.

The coordinator manages the broader task. It may interpret the request, decide which specialists are needed, assign work, track progress, and combine the results.

A role definition should make four things clear:

Question Example
What is the agent responsible for? Analyse payment failure patterns
What information does it need? Error metrics and service logs
Which tools can it use? Monitoring and log-search APIs
What must it return? Findings, evidence, and limitations

This avoids vague roles such as "expert agent" that do not define what the agent is actually expected to do.

The coordinator can also be a source of coupling. If every agent depends on it for every small decision, it may become a bottleneck. Keep its responsibilities focused on system-level coordination.

3. Agent Collaboration ModelsΒΆ

The collaboration model describes how agents exchange work and combine their outputs. The course introduces several common models.

Sequential collaborationΒΆ

One agent completes its task and passes the result to the next agent.

flowchart LR
    A["Research"] --> B["Analysis"] --> C["Writing"] --> D["Review"]

For example, a research agent gathers information, an analysis agent identifies the key findings, and a writing agent prepares a report.

Use this when each stage depends on the previous result. The trade-off is that the stages execute one after another.

Parallel collaborationΒΆ

Independent agents work at the same time, and an aggregator combines their outputs.

flowchart TD
    I["Research Request"] --> A["Market Agent"]
    I --> B["Technology Agent"]
    I --> C["Risk Agent"]
    A --> D["Aggregator"]
    B --> D
    C --> D
    D --> O["Combined Report"]

A market research task might be divided into market, technology, and risk analysis. These activities can proceed independently.

Parallel execution can reduce elapsed time, but the aggregator must account for incomplete, conflicting, or duplicated results.

Hub-and-spoke collaborationΒΆ

A central coordinator assigns work to specialists and collects their responses.

flowchart TD
    U["Request"] --> H["Coordinator"]
    H <--> A["Specialist A"]
    H <--> B["Specialist B"]
    H <--> C["Specialist C"]
    H --> R["Final Result"]

This model is useful when the required specialists depend on the request or when work needs central planning. The coordinator can become a bottleneck if too much responsibility is concentrated in it.

Interactive collaborationΒΆ

Agents exchange messages over multiple turns. One agent may ask another to clarify a result, challenge an assumption, or provide additional evidence.

For example, a writing agent can submit a draft to a reviewer. The reviewer identifies missing evidence, the writer revises the draft, and the reviewer checks it again.

This is useful when feedback improves quality, but the workflow needs a stopping condition. Otherwise, agents may continue exchanging messages without making meaningful progress.

These models can be combined. A coordinator might launch parallel research tasks, collect the findings, and then send the combined result through sequential review stages.

4. Agent SpecializationΒΆ

Specialization means giving an agent a focused capability and a clear boundary. It is not simply creating several agents with different names or personas.

Consider a document review system. A retrieval agent searches for relevant clauses. A compliance agent checks those clauses against approved requirements. A reporting agent explains the findings. Each agent has a distinct responsibility and needs only the information and tools required for that responsibility.

A useful handoff should be structured. Free-form text is flexible, but a predictable contract makes results easier for the application to validate.

Here is a small Python example using Pydantic:

from typing import Literal
from pydantic import BaseModel, Field


class AgentResult(BaseModel):
    task_id: str
    agent: str
    status: Literal["completed", "failed"]
    findings: list[str] = Field(default_factory=list)
    limitations: list[str] = Field(default_factory=list)


def process_result(result: AgentResult) -> str:
    if result.status == "failed":
        return f"Escalate task: {result.task_id}"

    if not result.findings:
        return f"Request more evidence: {result.task_id}"

    return f"Review findings from {result.agent}"

The receiving component can validate the status, inspect findings, and see whether the agent reported limitations. This is an application-level example, not a universal agent protocol.

Good specialization also requires balancing focus against coordination overhead. If a task is too small, creating a separate agent may add more handoffs and model calls than value. If responsibilities overlap heavily, it may be difficult to identify which agent owns a decision.

5. Multi-Agent System ArchitectureΒΆ

A multi-agent architecture needs more than a collection of prompts. It needs a way to assign tasks, manage context, exchange results, and determine whether the overall objective has been met.

flowchart TD
    U["Business Request"] --> C["Coordinator / Workflow"]
    C --> P["Task Planning"]
    P --> A["Specialist Agents"]
    A <--> T["Enterprise Tools and Services"]
    A --> V["Validate Results"]
    V --> C
    C --> D{"Objective satisfied?"}
    D -->|No| P
    D -->|Yes| S["Synthesize Final Result"]

The main responsibilities are:

  • Task planning: translate the objective into work that can be assigned.
  • Agent execution: perform the specialist tasks using permitted context and tools.
  • Communication and handoff: transfer task inputs, outputs, status, and relevant context.
  • Validation: check whether returned results are usable and sufficiently supported.
  • Synthesis: combine accepted results into an outcome that addresses the original request.

For production systems, the architecture should also define what happens when an agent fails, times out, returns invalid data, or cannot access a required service. The system should preserve task status and make partial completion visible.

A successful tool call is not proof that the business task is complete. The overall result still needs validation.

6. Enterprise Multi-Agent Use CasesΒΆ

Multi-agent designs are most useful when a business task naturally divides into distinct areas of expertise.

Market research

A research agent gathers sources, an analysis agent identifies trends, a critique agent reviews the reasoning, and a compiler produces the final report. Independent research streams may run in parallel.

Customer support

An intent agent identifies the customer's issue, a retrieval agent finds relevant policy or product information, and a response agent prepares an answer. A coordinator can route complex or unresolved cases for human review.

Legal contract review

A clause extraction agent identifies relevant contract sections, a compliance agent checks them against defined requirements, and a risk agent highlights clauses that need attention. A reporting agent consolidates the findings. The system should present source clauses and uncertainty for human review rather than treating generated conclusions as legal approval.

Payment incident investigation

A diagnostics agent examines application errors, an infrastructure agent checks service health, and a deployment agent reviews recent releases. An analyst combines the evidence and prepares a probable-cause assessment.

These examples illustrate the value of specialization, but they do not imply that every use case needs multiple agents. A conventional workflow or a single agent may be simpler and more reliable when the task is narrow and predictable.

Key TakeawaysΒΆ

  • A multi-agent system brings multiple agents together around individual or shared objectives.
  • Clear roles and responsibility boundaries make collaboration easier to manage.
  • Sequential, parallel, hub-and-spoke, and interactive models support different task dependencies.
  • Structured handoffs help validate results and expose limitations.
  • More agents introduce coordination cost and failure paths; they do not automatically provide scalability or fault tolerance.
  • Choose a multi-agent design when specialization provides measurable value over a simpler architecture.

What's Next?ΒΆ

The next chapter explores Agentic Workflow Design Patterns: how sequential, routing, parallelization, orchestrator-worker, and evaluator-optimizer patterns structure the execution of agentic tasks.

ReferencesΒΆ


🧭 Chapter Navigation¢

⬅️ Previous: 01. Agentic AI Fundamentals

πŸ“š Part: Agentic AI: Multi-Agent Systems Architecture, Orchestration, Collaboration

➑️ Next: 03β€”Agentic Workflow Design Patterns


Enterprise AI Engineering Handbook\ Building Production-Grade Enterprise AI Systems --- One Chapter at a Time.