How Multi-Agent Orchestration Works in OpenMAIC

OpenMAIC implements multi-agent orchestration through a single-pass, stateless LangGraph that alternates between a director node for agent selection and an agent-generate node for execution, requiring no server-side persistence.

The THU-MAIC/OpenMAIC repository provides a sophisticated approach to multi-agent orchestration using a stateless architecture built on LangGraph. Unlike traditional persistent workflow engines, this system encapsulates all orchestration state—agent configurations, turn counts, and whiteboard ledgers—within the request payload. This design enables horizontal scaling and simplifies deployment across distributed environments while maintaining complex conversational workflows between multiple AI agents.

Stateless LangGraph Architecture

OpenMAIC defines its execution topology in lib/orchestration/director-graph.ts through the createOrchestrationGraph function. The graph follows a deterministic path: START → director → agent_generate → END. This structure ensures that each request executes exactly one cycle of director routing followed by agent generation before returning control to the client.

The orchestration relies on OrchestratorState, a TypeScript interface that maintains mutable fields including currentAgent, turnCount, agentResponses, and whiteboardLedger, alongside immutable request data. Because the system never persists this state server-side, the client must serialize the returned state after each turn to drive subsequent interactions.

The Director Node: Routing Logic

The director node serves as the traffic controller for multi-agent conversations, implementing both deterministic fast-paths and LLM-based decision making.

Fast-Path Optimizations

When only one agent is available in the request configuration, the director bypasses the LLM entirely and dispatches that agent directly. This code-only path avoids unnecessary latency for single-agent scenarios, as implemented in lines 15-33 of lib/orchestration/director-graph.ts.

For the first turn of multi-agent discussions, the system supports a trigger agent fast-path. If triggerAgentId is specified in the request config, the director immediately selects that agent without consulting the LLM, optimizing for scenarios where the conversation starter is predetermined.

LLM-Based Decision Making

When multiple agents are available and no trigger agent is specified, the director constructs a system prompt via buildDirectorPrompt that describes available agents, conversation history, and the current whiteboard ledger. The LLM returns a structured decision that parseDirectorDecision parses into three possible outcomes:

• END – Terminate the discussion immediately
• USER – Cue the human user to speak next
• <agent-id> – Dispatch the specific agent by its identifier

This logic spans lines 61-88 in lib/orchestration/director-graph.ts, allowing complex routing decisions based on conversational context rather than hardcoded rules.

The Agent-Generate Node: Execution Pipeline

Once the director selects an agent, the agent-generate node executes a single generation pass that streams structured output back to the client.

Streaming and Structured Output Parsing

The node constructs a specialized system prompt through buildStructuredPrompt, embedding the agent's configuration, scene state, and discussion context. It then calls the language model via AISdkLangGraphAdapter.streamGenerate, receiving tokens that interleave natural text with executable actions.

To handle partial JSON fragments during streaming, OpenMAIC implements a partial-JSON parser in lib/orchestration/stateless-generate.ts (lines 35-77). The parser incrementally builds an ordered array containing objects with type: "text" or type: "action", preserving the original interleaving sequence. The utility functions parseStructuredChunk and finalizeParser manage trailing fragments and suppress raw JSON leakage, ensuring the client receives clean event streams even when the LLM output is chunked unpredictably.

Event-Driven Client Communication

The statelessGenerate function in lib/orchestration/stateless-generate.ts yields a stream of StatelessEvent objects that the client consumes in real time:

• thinking – Indicates director processing or agent loading
• text_delta – Streaming text content from the agent
• action – Structured action objects for whiteboard manipulation
• cue_user – Signal that the user should provide input
• done – Turn completion with final state
• error – Error handling with descriptive messages

After an agent completes generation, the graph records text previews, action counts, and whiteboard updates into agentResponses and whiteboardLedger, increments turnCount, and updates totalActions before determining whether to end or return to the director.

Practical Implementation Examples

Initiating a Stateless Multi-Agent Generation

import { statelessGenerate } from '@/lib/orchestration/stateless-generate';
import { createLanguageModel } from '@/lib/provider/openai';

const request = {
  messages: [...],
  storeState: {...},
  config: {
    agentIds: ['teacher', 'assistant'],
    triggerAgentId: 'teacher',
    discussionTopic: 'Photosynthesis',
    agentConfigs: [],
  },
  directorState: undefined,
  userProfile: { nickname: 'Bob' },
};

const lm = await createLanguageModel({ model: 'gpt-4o-mini' });
const abort = new AbortController();

for await (const event of statelessGenerate(request, abort.signal, lm)) {
  switch (event.type) {
    case 'text_delta':
      process.stdout.write(event.data.content);
      break;
    case 'action':
      console.log('[action]', event.data);
      break;
    case 'cue_user':
      console.log('Please reply...');
      break;
    case 'done':
      console.log('Turn finished', event.data);
      break;
  }
}

Adding Dynamically Generated Agents

const generatedAgent = {
  id: 'quiz-master',
  name: 'Quiz Master',
  avatar: '/avatars/quiz.png',
  allowedActions: ['question', 'answer'],
  systemPrompt: 'You run a quiz about biology.',
};

request.config.agentIds.push(generatedAgent.id);
request.config.agentConfigs = [generatedAgent];

Because buildInitialState copies agentConfigs into agentConfigOverrides, dynamically created agents are immediately discoverable by the director without server-side registration.

Manually Triggering Specific Agents

request.config.triggerAgentId = 'assistant';

Setting the triggerAgentId forces the director to bypass LLM deliberation and dispatch the specified agent on the next turn.

Summary

• Stateless Architecture: All orchestration state travels via the request payload, eliminating server-side persistence requirements and enabling horizontal scaling.
• Two-Node Graph: The LangGraph implementation alternates between a director node (routing) and an agent-generate node (execution) in lib/orchestration/director-graph.ts.
• Intelligent Routing: The director employs fast-paths for single agents and trigger agents, falling back to LLM-based decisions for complex multi-agent scenarios.
• Streaming Parser: Partial JSON parsing in lib/orchestration/stateless-generate.ts handles interleaved text and action tokens while preventing leakage of raw JSON to clients.
• Dynamic Extensibility: Agent configurations can be passed per-request via agentConfigs, allowing runtime agent creation without service restarts.

Frequently Asked Questions

What makes OpenMAIC's multi-agent orchestration stateless?

Every request to statelessGenerate carries its complete context—including agent configurations, conversation history, turn counts, and whiteboard state—within the StatelessChatRequest payload. The server processes each turn independently without maintaining session state, returning an updated OrchestratorState that the client stores for the next request. This design enables deployment across serverless environments and load-balanced clusters without sticky sessions.

How does the director decide which agent speaks next?

The director implements a decision hierarchy defined in lib/orchestration/director-graph.ts. If only one agent exists, it dispatches immediately. If a triggerAgentId is specified for the first turn, it uses that fast-path. Otherwise, it prompts the LLM with agent descriptions and conversation context via buildDirectorPrompt, then parses the response through parseDirectorDecision to select an agent, cue the user, or end the discussion.

Can agents be added dynamically without server restarts?

Yes. By populating the agentConfigs array in the request configuration, clients can inject new agent definitions at runtime. The buildInitialState function merges these overrides into the orchestration state, making them available to the director's routing logic immediately. This supports use cases where agents are generated on-the-fly based on user preferences or contextual requirements.

How does the system handle streaming JSON output from agents?

OpenMAIC uses a custom partial-JSON parser implemented in lib/orchestration/stateless-generate.ts that processes incremental chunks via parseStructuredChunk. The parser maintains an internal buffer for incomplete JSON fragments and uses finalizeParser to handle edge cases when streams terminate. This ensures that interleaved text and action tokens are emitted as discrete events (text_delta and action) without exposing raw JSON structures to the client.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}