How OpenMAIC Uses LangGraph for Multi-Agent Orchestration

OpenMAIC implements deterministic multi-agent coordination using LangGraph's StateGraph to route conversations between a director node and agent generation nodes in a single-pass, stateless architecture.

OpenMAIC is an open-source framework that leverages LangGraph to manage complex multi-agent chat workflows. The system implements a deterministic state machine where a director node decides which agent speaks next, followed by an agent generation node that streams structured responses back to the client. This architecture ensures predictable execution while maintaining flexibility for both single-agent and multi-agent scenarios without requiring persistent server-side state.

The Two-Stage Orchestration Architecture

OpenMAIC's LangGraph implementation centers on a strict two-node topology that separates decision-making from generation. This design enforces a single-round contract per request, eliminating the need for explicit turn limits while maintaining deterministic execution.

Director Node

The director node (implemented in lib/orchestration/director-graph.ts) determines which participant speaks next. It handles three distinct execution paths:

• Single-agent fast-path: Uses deterministic code without LLM calls when only one agent is configured.
• Trigger-agent shortcut: Immediately routes to a specified trigger agent on the first turn.
• LLM-driven selection: Invokes a language model via the AISdkLangGraphAdapter to select the next speaker based on conversation context.

The director streams reasoning events—specifically thinking and cue_user types—through LangGraph's custom stream mode via config.writer, allowing real-time visibility into the orchestration logic.

Agent-Generate Node

Once the director selects an agent, control passes to the agent-generate node. This node:

1. Constructs a system prompt using buildStructuredPrompt
2. Converts chat history into OpenAI-compatible message formats
3. Streams LLM output while parsing partial JSON chunks via parseStructuredChunk
4. Emits structured events including text_delta, action, and agent_end to the client

The partial-JSON parser interleaves text content with executable actions, enabling the front-end to render responses incrementally while preparing function calls.

Graph Topology and Execution Flow

The orchestration graph follows a strict linear topology with one conditional branch. As defined in lib/orchestration/director-graph.ts, the compiled StateGraph enforces the following execution contract:

START → director ──(end)──→ END
           │
           └─(next)→ agent_generate ──→ END

Each request executes at most one director → agent_generate cycle. The client serializes multiple requests to continue longer discussions, making the topology itself responsible for bounding conversation length rather than explicit maxTurns counters.

Building the Compiled Graph

The graph construction uses LangGraph's StateGraph class with explicit state annotations:

// lib/orchestration/director-graph.ts
export function createOrchestrationGraph() {
  const graph = new StateGraph(OrchestratorState)
    .addNode('director', directorNode)
    .addNode('agent_generate', agentGenerateNode)
    .addEdge(START, 'director')
    .addConditionalEdges('director', directorCondition, {
      agent_generate: 'agent_generate',
      [END]: END,
    })
    .addEdge('agent_generate', END);

  return graph.compile();
}

The directorCondition function evaluates the current state to determine whether to route to agent_generate or terminate the graph at END.

State Management with Annotations

OpenMAIC uses LangGraph's Annotation.Root system to define type-safe state schemas (lines 48-77 of director-graph.ts). The state separates immutable inputs from mutable execution fields:

Immutable inputs include:

• messages: Chat history array
• agents: Available agent configurations
• languageModel: The LLM instance provided per-request

Mutable execution fields include:

• currentAgentId: Tracks which agent is currently generating
• turnCount: Monitors conversation depth
• agentResponses: Accumulates outputs for whiteboard actions
• shouldEnd: Boolean flag for early termination

This immutable-by-default approach ensures that each request starts from a clean state defined entirely by the incoming StatelessChatRequest, supporting the framework's stateless architecture.

The Director Decision Logic

The director node implements sophisticated routing logic without sacrificing determinism. For single-agent scenarios, the director immediately routes to that agent without LLM invocation. For multi-agent scenarios, it first checks for a triggerAgentId in the request configuration—if present on turn zero, it fast-tracks that agent.

When neither shortcut applies, the director:

1. Summarizes conversation history (via conversation-summary.ts)
2. Constructs a director-specific prompt using prompt-builder.ts
3. Invokes the LLM through the AISdkLangGraphAdapter (lines 150-190)
4. Parses the decision via parseDirectorDecision
5. Streams the reasoning process through config.writer as thinking events before routing

Agent Generation and Streaming Execution

The agent-generate node handles the actual LLM interaction and response parsing. Located in lib/orchestration/stateless-generate.ts, this node:

1. Filters actions: Calls getEffectiveActions to determine which tools the current agent may invoke
2. Streams chunks: Consumes the LLM stream and feeds it to parseStructuredChunk
3. Maintains parser state: Uses createParserState() to track partial JSON across chunks
4. Emits events: Pushes StatelessEvent objects through config.writer for real-time client updates

Processing Streamed Responses

The partial-JSON parser handles interleaved text and action items:

import { parseStructuredChunk, createParserState } from '@/lib/orchestration/stateless-generate';

let parser = createParserState();
for await (const chunk of llmStream) {
  const parsed = parseStructuredChunk(chunk, parser);
  // parsed.textChunks contains incremental text
  // parsed.actions contains structured function calls
}

This approach allows agents to emit both conversational text and executable actions in a single streaming response, with the parser maintaining state across network chunks.

Stateless Request Handling

OpenMAIC treats each HTTP request as an isolated LangGraph execution. The entry point in lib/orchestration/stateless-generate.ts constructs fresh state instances for every invocation:

export async function generate(
  request: StatelessChatRequest,
  languageModel: LanguageModel,
  thinkingConfig?: ThinkingConfig,
) {
  const graph = createOrchestrationGraph();
  const initialState = buildInitialState(request, languageModel, thinkingConfig);
  return await graph.invoke(initialState);
}

The buildInitialState function (defined in director-graph.ts) hydrates the graph's initial context from the request payload, including any dynamically generated agent configurations. Because the compiled graph executes fresh with each request, the server maintains no persistent chat state between turns—all conversation history must be provided by the client in each StatelessChatRequest.

Summary

• OpenMAIC uses LangGraph's StateGraph to implement a deterministic two-node orchestration consisting of a director and an agent-generate node.
• The graph topology explicitly bounds execution to at most one agent turn per request, with the client managing multi-turn conversations through subsequent requests.
• State management relies on Annotation.Root to separate immutable request data from mutable execution state, enabling completely stateless server operation.
• The director node implements fast-paths for single-agent and trigger-agent scenarios, falling back to LLM-driven selection only when necessary.
• Streaming architecture uses partial-JSON parsing to interleave text deltas and structured actions, emitting real-time events via LangGraph's stream configuration.
• All core logic resides in lib/orchestration/director-graph.ts and lib/orchestration/stateless-generate.ts, with adapters bridging to the Vercel AI SDK for model compatibility.

Frequently Asked Questions

How does OpenMAIC handle conversation history without server-side state?

OpenMAIC maintains statelessness by requiring the client to provide complete conversation history in each StatelessChatRequest. The buildInitialState function hydrates the LangGraph state from this request payload, and the graph executes as a single pass. When the graph reaches the END node, the server returns the response and discards all state, requiring the client to include previous messages in subsequent requests.

What determines whether the director uses an LLM or code-based routing?

The director node (lines 91-122 in director-graph.ts) uses deterministic code paths when only one agent is configured or when a triggerAgentId is specified for the first turn. It invokes an LLM only when multiple agents are available and no trigger override exists. This hybrid approach minimizes latency for simple scenarios while maintaining flexibility for complex multi-agent coordination.

How does the partial-JSON parser handle malformed or incomplete chunks?

The parseStructuredChunk utility (found in stateless-generate.ts) maintains accumulator state via createParserState(), buffering incomplete JSON across stream chunks. It validates structures incrementally, emitting text_delta events for valid content and action objects only when complete JSON objects are received. If a chunk splits mid-object, the parser retains the partial state until the next chunk arrives, ensuring robust handling of network fragmentation.

Can the orchestration graph handle tool calling or function execution?

Yes, the agent-generate node filters available actions through getEffectiveActions and parses structured tool calls via the partial-JSON parser. The streaming architecture emits action events distinct from text_delta events, allowing the client to handle function execution while the agent continues generating text. However, the current topology confines tool results to the single turn—complex multi-step tool workflows would require client-side orchestration to feed results back into subsequent requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}