How LangGraph's Custom Stream Mode Powers Real-Time SSE Delivery in OpenMAIC

LangGraph's custom stream mode enables OpenMAIC to stream AI interaction events—text, actions, and thinking states—directly from graph nodes to the client via Server-Sent Events without waiting for full execution completion.

OpenMAIC is an open-source AI tutoring platform that uses LangGraph to orchestrate multi-agent conversations. By leveraging LangGraph's custom stream mode, the system pushes StatelessEvent objects in real-time through an SSE connection, allowing the frontend to render tutor responses progressively as they are generated.

The Architecture of Real-Time Streaming

OpenMAIC's streaming architecture rests on a stateless design where the orchestration graph runs in LangGraph while events flow outward to the client.

Stateless Events and the SSE Protocol

According to the OpenMAIC source code, events follow the StatelessEvent type defined in lib/types/chat.ts. These events carry a discriminated union type field that can be 'text', 'action', 'thinking', or 'cue_user', along with minimal payloads required for UI rendering.

Because the graph maintains no server-side state between requests, all context passes through the initial state object. The custom stream mode breaks the traditional request-response cycle by allowing nodes to emit events incrementally:

• Text chunks arrive as the LLM generates tokens
• Action events signal whiteboard updates or tool calls
• Thinking events indicate which agent is currently processing

The LangGraph Configuration

The orchestration graph is built using LangGraph's StateGraph class. Unlike standard graph execution that returns a final state object, OpenMAIC configures the graph with a custom writer function accessible via config.writer in every node.

Implementing Custom Stream Mode in OpenMAIC

The implementation spans multiple files in the lib/orchestration/ directory, with each node using config.writer to push StatelessEvent chunks directly to the HTTP response.

Director Node Streaming (director-graph.ts)

The director node, located in lib/orchestration/director-graph.ts, decides which agent speaks next and streams status updates to the client. As documented in lines 20-22, this module "Uses LangGraph's custom stream mode: each node pushes StatelessEvent chunks via config.writer() for real-time SSE delivery."

// lib/orchestration/director-graph.ts
async function directorNode(state, config): Promise<Partial<OrchestratorStateType>> {
  const rawWrite = config.writer as (chunk: StatelessEvent) => void;
  const write = (chunk: StatelessEvent) => {
    try { rawWrite(chunk); } catch { /* controller closed */ }
  };

  // ... decision logic ...

  // Push a streaming event to the client
  write({ type: 'thinking', data: { stage: 'director' } });
  // … more logic …
}

The write function wraps config.writer to safely handle cases where the client disconnects mid-stream, ensuring the server remains robust during long-running generations.

Agent Generation Streaming (stateless-generate.ts)

The agent generation node handles LLM output parsing and incrementally streams text chunks. The lib/orchestration/stateless-generate.ts module explicitly notes that "Events are streamed via LangGraph's custom stream mode" (lines 18-19).

// lib/orchestration/stateless-generate.ts
export function parseStructuredChunk(chunk: string, state: ParserState): ParseResult {
  // … accumulate chunk, parse partial JSON …
  if (newText) {
    // Emit a text chunk as soon as it becomes available
    result.textChunks.push(newText);
    result.ordered.push({ type: 'text', index: result.textChunks.length - 1 });
  }
  // The caller (agent-generate node) will forward these
  // via config.writer() to the SSE response.
}

This parsing approach allows the system to extract structured data from the LLM's partial JSON output while simultaneously streaming readable text segments to the user interface.

Wiring the Writer to the HTTP Response

The connection between LangGraph's internal streaming mechanism and the client's SSE connection happens at the API layer. The runtime injects a writer function that bridges graph execution to the HTTP response:

// Simplified from render-service/api.ts
export async function handleChat(req, res) {
  const graph = createOrchestrationGraph();          // LangGraph graph
  const sse = new SSE(res);                           // npm sse library
  const config = { writer: (e) => sse.send(e) };       // custom stream mode
  await graph.run(initialState, config);              // runs graph, streams events
}

When graph.run() executes, every call to config.writer() within the director or agent nodes immediately serializes the StatelessEvent and transmits it over the SSE connection. This creates a continuous flow of real-time updates while maintaining the stateless architecture required for scalable deployment.

Summary

• LangGraph's custom stream mode provides a config.writer function that allows individual graph nodes to emit events during execution rather than only returning final results.
• OpenMAIC uses this mechanism in lib/orchestration/director-graph.ts and lib/orchestration/stateless-generate.ts to stream StatelessEvent objects containing text, actions, and thinking states.
• The stateless architecture ensures all context travels with the request, while SSE delivery happens through incrementally written chunks.
• Real-time tutoring interactions rely on this streaming pipeline to display AI-generated content as it is produced, rather than buffering until completion.

Frequently Asked Questions

What is LangGraph's custom stream mode?

LangGraph's custom stream mode is an execution configuration that injects a writer function (config.writer) into each graph node. Instead of nodes returning only state updates at the end of execution, they can call this writer repeatedly to push custom event objects to an external consumer. In OpenMAIC, this enables real-time SSE delivery of tutoring interactions.

How does OpenMAIC maintain statelessness while streaming real-time events?

OpenMAIC maintains statelessness by passing all conversation context through the initial state object when the graph starts. The StatelessEvent type defined in lib/types/chat.ts contains only the minimal data needed for UI updates, with no reference to server-side session state. Because the graph nodes receive config.writer as an injection, they can emit events without maintaining connections between requests.

What types of events does OpenMAIC stream via SSE?

OpenMAIC streams four primary event types defined in the StatelessEvent union: 'text' for LLM-generated content, 'action' for whiteboard or tool operations, 'thinking' to indicate which processing stage is active, and 'cue_user' for UI prompts. Each event type carries a specific payload structure optimized for frontend rendering.

How does the director node differ from agent nodes in streaming behavior?

The director node in lib/orchestration/director-graph.ts primarily emits 'thinking' events to signal decision-making progress, while agent nodes in the generation pipeline stream 'text' chunks and 'action' events as they parse LLM output. Both use the same config.writer mechanism, but the director focuses on orchestration status while agents handle content generation streaming.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}