# How LangGraph's Custom Stream Mode Powers Real-Time SSE Delivery in OpenMAIC

> Discover how LangGraph's custom stream mode empowers OpenMAIC to deliver real-time SSE updates from graph nodes, enhancing AI interaction delivery.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-04

---

**LangGraph's custom stream mode enables OpenMAIC to stream AI interaction events—text, actions, and thinking states—directly from graph nodes to the client via Server-Sent Events without waiting for full execution completion.**

OpenMAIC is an open-source AI tutoring platform that uses LangGraph to orchestrate multi-agent conversations. By leveraging LangGraph's custom stream mode, the system pushes `StatelessEvent` objects in real-time through an SSE connection, allowing the frontend to render tutor responses progressively as they are generated.

## The Architecture of Real-Time Streaming

OpenMAIC's streaming architecture rests on a stateless design where the orchestration graph runs in LangGraph while events flow outward to the client.

### Stateless Events and the SSE Protocol

According to the OpenMAIC source code, events follow the `StatelessEvent` type defined in [`lib/types/chat.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/chat.ts). These events carry a discriminated union type field that can be `'text'`, `'action'`, `'thinking'`, or `'cue_user'`, along with minimal payloads required for UI rendering.

Because the graph maintains no server-side state between requests, all context passes through the initial state object. The custom stream mode breaks the traditional request-response cycle by allowing nodes to emit events incrementally:

- **Text chunks** arrive as the LLM generates tokens
- **Action events** signal whiteboard updates or tool calls  
- **Thinking events** indicate which agent is currently processing

### The LangGraph Configuration

The orchestration graph is built using LangGraph's `StateGraph` class. Unlike standard graph execution that returns a final state object, OpenMAIC configures the graph with a custom writer function accessible via `config.writer` in every node.

## Implementing Custom Stream Mode in OpenMAIC

The implementation spans multiple files in the `lib/orchestration/` directory, with each node using `config.writer` to push `StatelessEvent` chunks directly to the HTTP response.

### Director Node Streaming (director-graph.ts)

The director node, located in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts), decides which agent speaks next and streams status updates to the client. As documented in lines 20-22, this module "Uses LangGraph's custom stream mode: each node pushes StatelessEvent chunks via config.writer() for real-time SSE delivery."

```typescript
// lib/orchestration/director-graph.ts
async function directorNode(state, config): Promise<Partial<OrchestratorStateType>> {
  const rawWrite = config.writer as (chunk: StatelessEvent) => void;
  const write = (chunk: StatelessEvent) => {
    try { rawWrite(chunk); } catch { /* controller closed */ }
  };

  // ... decision logic ...

  // Push a streaming event to the client
  write({ type: 'thinking', data: { stage: 'director' } });
  // … more logic …
}

```

The `write` function wraps `config.writer` to safely handle cases where the client disconnects mid-stream, ensuring the server remains robust during long-running generations.

### Agent Generation Streaming (stateless-generate.ts)

The agent generation node handles LLM output parsing and incrementally streams text chunks. The [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) module explicitly notes that "Events are streamed via LangGraph's custom stream mode" (lines 18-19).

```typescript
// lib/orchestration/stateless-generate.ts
export function parseStructuredChunk(chunk: string, state: ParserState): ParseResult {
  // … accumulate chunk, parse partial JSON …
  if (newText) {
    // Emit a text chunk as soon as it becomes available
    result.textChunks.push(newText);
    result.ordered.push({ type: 'text', index: result.textChunks.length - 1 });
  }
  // The caller (agent-generate node) will forward these
  // via config.writer() to the SSE response.
}

```

This parsing approach allows the system to extract structured data from the LLM's partial JSON output while simultaneously streaming readable text segments to the user interface.

## Wiring the Writer to the HTTP Response

The connection between LangGraph's internal streaming mechanism and the client's SSE connection happens at the API layer. The runtime injects a writer function that bridges graph execution to the HTTP response:

```typescript
// Simplified from render-service/api.ts
export async function handleChat(req, res) {
  const graph = createOrchestrationGraph();          // LangGraph graph
  const sse = new SSE(res);                           // npm sse library
  const config = { writer: (e) => sse.send(e) };       // custom stream mode
  await graph.run(initialState, config);              // runs graph, streams events
}

```

When `graph.run()` executes, every call to `config.writer()` within the director or agent nodes immediately serializes the `StatelessEvent` and transmits it over the SSE connection. This creates a continuous flow of real-time updates while maintaining the stateless architecture required for scalable deployment.

## Summary

- **LangGraph's custom stream mode** provides a `config.writer` function that allows individual graph nodes to emit events during execution rather than only returning final results.
- OpenMAIC uses this mechanism in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts) and [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) to stream `StatelessEvent` objects containing text, actions, and thinking states.
- The **stateless architecture** ensures all context travels with the request, while SSE delivery happens through incrementally written chunks.
- Real-time tutoring interactions rely on this streaming pipeline to display AI-generated content as it is produced, rather than buffering until completion.

## Frequently Asked Questions

### What is LangGraph's custom stream mode?

LangGraph's custom stream mode is an execution configuration that injects a writer function (`config.writer`) into each graph node. Instead of nodes returning only state updates at the end of execution, they can call this writer repeatedly to push custom event objects to an external consumer. In OpenMAIC, this enables real-time SSE delivery of tutoring interactions.

### How does OpenMAIC maintain statelessness while streaming real-time events?

OpenMAIC maintains statelessness by passing all conversation context through the initial state object when the graph starts. The `StatelessEvent` type defined in [`lib/types/chat.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/chat.ts) contains only the minimal data needed for UI updates, with no reference to server-side session state. Because the graph nodes receive `config.writer` as an injection, they can emit events without maintaining connections between requests.

### What types of events does OpenMAIC stream via SSE?

OpenMAIC streams four primary event types defined in the `StatelessEvent` union: `'text'` for LLM-generated content, `'action'` for whiteboard or tool operations, `'thinking'` to indicate which processing stage is active, and `'cue_user'` for UI prompts. Each event type carries a specific payload structure optimized for frontend rendering.

### How does the director node differ from agent nodes in streaming behavior?

The **director node** in [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts) primarily emits `'thinking'` events to signal decision-making progress, while **agent nodes** in the generation pipeline stream `'text'` chunks and `'action'` events as they parse LLM output. Both use the same `config.writer` mechanism, but the director focuses on orchestration status while agents handle content generation streaming.