# How OpenMAIC Uses LangGraph for Multi-Agent Orchestration

> Discover how OpenMAIC leverages LangGraph for efficient multi-agent orchestration. Learn about its stateless architecture and single-pass conversation routing for deterministic coordination.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-12

---

**OpenMAIC implements deterministic multi-agent coordination using LangGraph's StateGraph to route conversations between a director node and agent generation nodes in a single-pass, stateless architecture.**

OpenMAIC is an open-source framework that leverages LangGraph to manage complex multi-agent chat workflows. The system implements a deterministic state machine where a director node decides which agent speaks next, followed by an agent generation node that streams structured responses back to the client. This architecture ensures predictable execution while maintaining flexibility for both single-agent and multi-agent scenarios without requiring persistent server-side state.

## The Two-Stage Orchestration Architecture

OpenMAIC's LangGraph implementation centers on a strict two-node topology that separates **decision-making** from **generation**. This design enforces a single-round contract per request, eliminating the need for explicit turn limits while maintaining deterministic execution.

### Director Node

The **director node** (implemented in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts)) determines which participant speaks next. It handles three distinct execution paths:

- **Single-agent fast-path**: Uses deterministic code without LLM calls when only one agent is configured.
- **Trigger-agent shortcut**: Immediately routes to a specified trigger agent on the first turn.
- **LLM-driven selection**: Invokes a language model via the `AISdkLangGraphAdapter` to select the next speaker based on conversation context.

The director streams reasoning events—specifically `thinking` and `cue_user` types—through LangGraph's custom stream mode via `config.writer`, allowing real-time visibility into the orchestration logic.

### Agent-Generate Node

Once the director selects an agent, control passes to the **agent-generate node**. This node:

1. Constructs a system prompt using `buildStructuredPrompt`
2. Converts chat history into OpenAI-compatible message formats
3. Streams LLM output while parsing partial JSON chunks via `parseStructuredChunk`
4. Emits structured events including `text_delta`, `action`, and `agent_end` to the client

The partial-JSON parser interleaves text content with executable actions, enabling the front-end to render responses incrementally while preparing function calls.

## Graph Topology and Execution Flow

The orchestration graph follows a strict linear topology with one conditional branch. As defined in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts), the compiled StateGraph enforces the following execution contract:

```

START → director ──(end)──→ END
           │
           └─(next)→ agent_generate ──→ END

```

Each request executes at most one `director → agent_generate` cycle. The client serializes multiple requests to continue longer discussions, making the topology itself responsible for bounding conversation length rather than explicit `maxTurns` counters.

### Building the Compiled Graph

The graph construction uses LangGraph's `StateGraph` class with explicit state annotations:

```typescript
// lib/orchestration/director-graph.ts
export function createOrchestrationGraph() {
  const graph = new StateGraph(OrchestratorState)
    .addNode('director', directorNode)
    .addNode('agent_generate', agentGenerateNode)
    .addEdge(START, 'director')
    .addConditionalEdges('director', directorCondition, {
      agent_generate: 'agent_generate',
      [END]: END,
    })
    .addEdge('agent_generate', END);

  return graph.compile();
}

```

The `directorCondition` function evaluates the current state to determine whether to route to `agent_generate` or terminate the graph at `END`.

## State Management with Annotations

OpenMAIC uses LangGraph's `Annotation.Root` system to define type-safe state schemas (lines 48-77 of [`director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/director-graph.ts)). The state separates immutable inputs from mutable execution fields:

**Immutable inputs** include:
- `messages`: Chat history array
- `agents`: Available agent configurations
- `languageModel`: The LLM instance provided per-request

**Mutable execution fields** include:
- `currentAgentId`: Tracks which agent is currently generating
- `turnCount`: Monitors conversation depth
- `agentResponses`: Accumulates outputs for whiteboard actions
- `shouldEnd`: Boolean flag for early termination

This immutable-by-default approach ensures that each request starts from a clean state defined entirely by the incoming `StatelessChatRequest`, supporting the framework's stateless architecture.

## The Director Decision Logic

The director node implements sophisticated routing logic without sacrificing determinism. For **single-agent scenarios**, the director immediately routes to that agent without LLM invocation. For **multi-agent scenarios**, it first checks for a `triggerAgentId` in the request configuration—if present on turn zero, it fast-tracks that agent.

When neither shortcut applies, the director:
1. Summarizes conversation history (via [`conversation-summary.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/conversation-summary.ts))
2. Constructs a director-specific prompt using [`prompt-builder.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/prompt-builder.ts)
3. Invokes the LLM through the `AISdkLangGraphAdapter` (lines 150-190)
4. Parses the decision via `parseDirectorDecision`
5. Streams the reasoning process through `config.writer` as `thinking` events before routing

## Agent Generation and Streaming Execution

The agent-generate node handles the actual LLM interaction and response parsing. Located in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts), this node:

1. **Filters actions**: Calls `getEffectiveActions` to determine which tools the current agent may invoke
2. **Streams chunks**: Consumes the LLM stream and feeds it to `parseStructuredChunk`
3. **Maintains parser state**: Uses `createParserState()` to track partial JSON across chunks
4. **Emits events**: Pushes `StatelessEvent` objects through `config.writer` for real-time client updates

### Processing Streamed Responses

The partial-JSON parser handles interleaved text and action items:

```typescript
import { parseStructuredChunk, createParserState } from '@/lib/orchestration/stateless-generate';

let parser = createParserState();
for await (const chunk of llmStream) {
  const parsed = parseStructuredChunk(chunk, parser);
  // parsed.textChunks contains incremental text
  // parsed.actions contains structured function calls
}

```

This approach allows agents to emit both conversational text and executable actions in a single streaming response, with the parser maintaining state across network chunks.

## Stateless Request Handling

OpenMAIC treats each HTTP request as an isolated LangGraph execution. The entry point in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) constructs fresh state instances for every invocation:

```typescript
export async function generate(
  request: StatelessChatRequest,
  languageModel: LanguageModel,
  thinkingConfig?: ThinkingConfig,
) {
  const graph = createOrchestrationGraph();
  const initialState = buildInitialState(request, languageModel, thinkingConfig);
  return await graph.invoke(initialState);
}

```

The `buildInitialState` function (defined in [`director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/director-graph.ts)) hydrates the graph's initial context from the request payload, including any dynamically generated agent configurations. Because the compiled graph executes fresh with each request, the server maintains no persistent chat state between turns—all conversation history must be provided by the client in each `StatelessChatRequest`.

## Summary

- OpenMAIC uses **LangGraph's StateGraph** to implement a deterministic two-node orchestration consisting of a director and an agent-generate node.
- The graph topology explicitly bounds execution to **at most one agent turn per request**, with the client managing multi-turn conversations through subsequent requests.
- **State management** relies on `Annotation.Root` to separate immutable request data from mutable execution state, enabling completely stateless server operation.
- The **director node** implements fast-paths for single-agent and trigger-agent scenarios, falling back to LLM-driven selection only when necessary.
- **Streaming architecture** uses partial-JSON parsing to interleave text deltas and structured actions, emitting real-time events via LangGraph's stream configuration.
- All core logic resides in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts) and [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts), with adapters bridging to the Vercel AI SDK for model compatibility.

## Frequently Asked Questions

### How does OpenMAIC handle conversation history without server-side state?

OpenMAIC maintains statelessness by requiring the client to provide complete conversation history in each `StatelessChatRequest`. The `buildInitialState` function hydrates the LangGraph state from this request payload, and the graph executes as a single pass. When the graph reaches the `END` node, the server returns the response and discards all state, requiring the client to include previous messages in subsequent requests.

### What determines whether the director uses an LLM or code-based routing?

The director node (lines 91-122 in [`director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/director-graph.ts)) uses deterministic code paths when only one agent is configured or when a `triggerAgentId` is specified for the first turn. It invokes an LLM only when multiple agents are available and no trigger override exists. This hybrid approach minimizes latency for simple scenarios while maintaining flexibility for complex multi-agent coordination.

### How does the partial-JSON parser handle malformed or incomplete chunks?

The `parseStructuredChunk` utility (found in [`stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/stateless-generate.ts)) maintains accumulator state via `createParserState()`, buffering incomplete JSON across stream chunks. It validates structures incrementally, emitting `text_delta` events for valid content and `action` objects only when complete JSON objects are received. If a chunk splits mid-object, the parser retains the partial state until the next chunk arrives, ensuring robust handling of network fragmentation.

### Can the orchestration graph handle tool calling or function execution?

Yes, the agent-generate node filters available actions through `getEffectiveActions` and parses structured tool calls via the partial-JSON parser. The streaming architecture emits `action` events distinct from `text_delta` events, allowing the client to handle function execution while the agent continues generating text. However, the current topology confines tool results to the single turn—complex multi-step tool workflows would require client-side orchestration to feed results back into subsequent requests.