# How Multi-Agent Orchestration Works in OpenMAIC

> Discover how OpenMAIC orchestrates multi-agent systems using a stateless LangGraph for efficient, serverless agent execution and selection. Learn the architecture today.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-09-01

---

**OpenMAIC implements multi-agent orchestration through a single-pass, stateless LangGraph that alternates between a director node for agent selection and an agent-generate node for execution, requiring no server-side persistence.**

The [THU-MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) repository provides a sophisticated approach to **multi-agent orchestration** using a stateless architecture built on LangGraph. Unlike traditional persistent workflow engines, this system encapsulates all orchestration state—agent configurations, turn counts, and whiteboard ledgers—within the request payload. This design enables horizontal scaling and simplifies deployment across distributed environments while maintaining complex conversational workflows between multiple AI agents.

## Stateless LangGraph Architecture

OpenMAIC defines its execution topology in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts) through the `createOrchestrationGraph` function. The graph follows a deterministic path: **START → director → agent_generate → END**. This structure ensures that each request executes exactly one cycle of director routing followed by agent generation before returning control to the client.

The orchestration relies on `OrchestratorState`, a TypeScript interface that maintains mutable fields including `currentAgent`, `turnCount`, `agentResponses`, and `whiteboardLedger`, alongside immutable request data. Because the system never persists this state server-side, the client must serialize the returned state after each turn to drive subsequent interactions.

## The Director Node: Routing Logic

The director node serves as the traffic controller for multi-agent conversations, implementing both deterministic fast-paths and LLM-based decision making.

### Fast-Path Optimizations

When only one agent is available in the request configuration, the director bypasses the LLM entirely and dispatches that agent directly. This code-only path avoids unnecessary latency for single-agent scenarios, as implemented in lines 15-33 of [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts).

For the first turn of multi-agent discussions, the system supports a **trigger agent** fast-path. If `triggerAgentId` is specified in the request config, the director immediately selects that agent without consulting the LLM, optimizing for scenarios where the conversation starter is predetermined.

### LLM-Based Decision Making

When multiple agents are available and no trigger agent is specified, the director constructs a system prompt via `buildDirectorPrompt` that describes available agents, conversation history, and the current whiteboard ledger. The LLM returns a structured decision that `parseDirectorDecision` parses into three possible outcomes:

- **END** – Terminate the discussion immediately
- **USER** – Cue the human user to speak next
- **`<agent-id>`** – Dispatch the specific agent by its identifier

This logic spans lines 61-88 in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts), allowing complex routing decisions based on conversational context rather than hardcoded rules.

## The Agent-Generate Node: Execution Pipeline

Once the director selects an agent, the `agent-generate` node executes a single generation pass that streams structured output back to the client.

### Streaming and Structured Output Parsing

The node constructs a specialized system prompt through `buildStructuredPrompt`, embedding the agent's configuration, scene state, and discussion context. It then calls the language model via `AISdkLangGraphAdapter.streamGenerate`, receiving tokens that interleave natural text with executable actions.

To handle partial JSON fragments during streaming, OpenMAIC implements a **partial-JSON parser** in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) (lines 35-77). The parser incrementally builds an `ordered` array containing objects with `type: "text"` or `type: "action"`, preserving the original interleaving sequence. The utility functions `parseStructuredChunk` and `finalizeParser` manage trailing fragments and suppress raw JSON leakage, ensuring the client receives clean event streams even when the LLM output is chunked unpredictably.

## Event-Driven Client Communication

The `statelessGenerate` function in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) yields a stream of `StatelessEvent` objects that the client consumes in real time:

- **`thinking`** – Indicates director processing or agent loading
- **`text_delta`** – Streaming text content from the agent
- **`action`** – Structured action objects for whiteboard manipulation
- **`cue_user`** – Signal that the user should provide input
- **`done`** – Turn completion with final state
- **`error`** – Error handling with descriptive messages

After an agent completes generation, the graph records text previews, action counts, and whiteboard updates into `agentResponses` and `whiteboardLedger`, increments `turnCount`, and updates `totalActions` before determining whether to end or return to the director.

## Practical Implementation Examples

### Initiating a Stateless Multi-Agent Generation

```typescript
import { statelessGenerate } from '@/lib/orchestration/stateless-generate';
import { createLanguageModel } from '@/lib/provider/openai';

const request = {
  messages: [...],
  storeState: {...},
  config: {
    agentIds: ['teacher', 'assistant'],
    triggerAgentId: 'teacher',
    discussionTopic: 'Photosynthesis',
    agentConfigs: [],
  },
  directorState: undefined,
  userProfile: { nickname: 'Bob' },
};

const lm = await createLanguageModel({ model: 'gpt-4o-mini' });
const abort = new AbortController();

for await (const event of statelessGenerate(request, abort.signal, lm)) {
  switch (event.type) {
    case 'text_delta':
      process.stdout.write(event.data.content);
      break;
    case 'action':
      console.log('[action]', event.data);
      break;
    case 'cue_user':
      console.log('Please reply...');
      break;
    case 'done':
      console.log('Turn finished', event.data);
      break;
  }
}

```

### Adding Dynamically Generated Agents

```typescript
const generatedAgent = {
  id: 'quiz-master',
  name: 'Quiz Master',
  avatar: '/avatars/quiz.png',
  allowedActions: ['question', 'answer'],
  systemPrompt: 'You run a quiz about biology.',
};

request.config.agentIds.push(generatedAgent.id);
request.config.agentConfigs = [generatedAgent];

```

Because `buildInitialState` copies `agentConfigs` into `agentConfigOverrides`, dynamically created agents are immediately discoverable by the director without server-side registration.

### Manually Triggering Specific Agents

```typescript
request.config.triggerAgentId = 'assistant';

```

Setting the `triggerAgentId` forces the director to bypass LLM deliberation and dispatch the specified agent on the next turn.

## Summary

- **Stateless Architecture**: All orchestration state travels via the request payload, eliminating server-side persistence requirements and enabling horizontal scaling.
- **Two-Node Graph**: The LangGraph implementation alternates between a director node (routing) and an agent-generate node (execution) in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts).
- **Intelligent Routing**: The director employs fast-paths for single agents and trigger agents, falling back to LLM-based decisions for complex multi-agent scenarios.
- **Streaming Parser**: Partial JSON parsing in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) handles interleaved text and action tokens while preventing leakage of raw JSON to clients.
- **Dynamic Extensibility**: Agent configurations can be passed per-request via `agentConfigs`, allowing runtime agent creation without service restarts.

## Frequently Asked Questions

### What makes OpenMAIC's multi-agent orchestration stateless?

Every request to `statelessGenerate` carries its complete context—including agent configurations, conversation history, turn counts, and whiteboard state—within the `StatelessChatRequest` payload. The server processes each turn independently without maintaining session state, returning an updated `OrchestratorState` that the client stores for the next request. This design enables deployment across serverless environments and load-balanced clusters without sticky sessions.

### How does the director decide which agent speaks next?

The director implements a decision hierarchy defined in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts). If only one agent exists, it dispatches immediately. If a `triggerAgentId` is specified for the first turn, it uses that fast-path. Otherwise, it prompts the LLM with agent descriptions and conversation context via `buildDirectorPrompt`, then parses the response through `parseDirectorDecision` to select an agent, cue the user, or end the discussion.

### Can agents be added dynamically without server restarts?

Yes. By populating the `agentConfigs` array in the request configuration, clients can inject new agent definitions at runtime. The `buildInitialState` function merges these overrides into the orchestration state, making them available to the director's routing logic immediately. This supports use cases where agents are generated on-the-fly based on user preferences or contextual requirements.

### How does the system handle streaming JSON output from agents?

OpenMAIC uses a custom partial-JSON parser implemented in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) that processes incremental chunks via `parseStructuredChunk`. The parser maintains an internal buffer for incomplete JSON fragments and uses `finalizeParser` to handle edge cases when streams terminate. This ensures that interleaved text and action tokens are emitted as discrete events (`text_delta` and `action`) without exposing raw JSON structures to the client.