# Handling Interleaved Structured JSON Actions and Freeform Text in OpenMAIC's Stateless Generate Process

> OpenMAIC's stateless generate process handles interleaved JSON actions and freeform text via a streaming parser. Learn how it extracts complete items and preserves order.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-04

---

**OpenMAIC processes interleaved structured JSON actions and freeform text through a streaming JSON array parser that incrementally extracts complete items while preserving their original order and gracefully handling malformed output.**

OpenMAIC's stateless generate pipeline enables large language models to emit mixed content types that combine silent tool executions with natural language responses. The system expects the LLM to output a single JSON array containing alternating objects that represent either executable actions or displayable text, processed in real-time through a robust streaming parser implemented in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts).

## The Interleaved JSON Array Format

The stateless generate process requires the LLM to structure its output as a JSON array containing two distinct object shapes that alternate throughout the stream:

- **Action objects**: `{"type":"action","name":"…","params":{…}}` (or the legacy `{"tool_name":"…","parameters":{…}}` format) represent silent tool calls that the client executes without displaying to the user.
- **Text objects**: `{"type":"text","content":"…"}` contain natural teacher speech meant for direct display.

This interleaved structure allows the model to switch between explaining concepts and triggering interface actions within a single generation stream.

## Streaming Parser Implementation

The core parsing logic resides in the `parseStructuredChunk` function, which maintains state across streaming chunks to build a complete parse tree incrementally.

### Buffer Management and Prefix Stripping

As LLM chunks arrive, the parser accumulates them in a buffer (lines 48‑59). The implementation specifically skips any prefix content before the opening `[` character, such as markdown code fences or explanatory text, ensuring that parsing begins at the actual JSON array boundary (lines 50‑58).

### Malformed JSON Recovery

To handle incomplete or malformed JSON common in streaming scenarios, the parser employs a dual-recovery strategy (lines 65‑78). It first attempts repair using **`jsonrepair`**, and if that fails, falls back to **`partial‑json`**. This robust approach allows the system to extract usable data even when the LLM produces malformed syntax or the stream is interrupted.

### Deterministic Item Ordering

When items are successfully parsed, the `emitItem` function (lines 75‑77) records each new object in a **`ParseResult`** structure. Crucially, the parser maintains an **`ordered`** array that preserves the exact sequence of text and action objects as they appeared in the original LLM output, ensuring the client can render content and execute actions in the correct chronological order.

## Processing Complete and Partial Items

The parser distinguishes between fully complete items and partially received content to enable low-latency streaming.

### Emitting Fully Formed Items

After a successful JSON parse, the code determines which array items are fully complete using the `completeUpTo` index (lines 86‑101). Only complete items are emitted immediately, while incomplete fragments remain in the buffer awaiting additional chunks. This prevents the system from prematurely processing truncated actions or cut-off text.

### Streaming Partial Text Deltas

For text items that are partially received but still incomplete, the parser implements delta streaming to minimize latency. It tracks the length of text already emitted via `lastPartialTextLength` (lines 34‑44) and streams only the newly available characters to the client. This approach allows the UI to display text in real-time without waiting for the entire JSON object to finish.

### Detecting Stream Completion

When the parser encounters the closing `]` character that terminates the JSON array, it sets `state.isDone` to true and marks the result as complete (lines 46‑52). This signals to the `statelessGenerate` async generator that the stream has finished and no further chunks are expected.

## Error Recovery and Finalization

If the LLM output never produces a top-level JSON array, the **`finalizeParser`** function (lines 57‑78) attempts to recover any plain-text objects with `type:"text"` that it can extract from the debris. The function suppresses stray JSON fragments that cannot be parsed and logs these suppressions for debugging, ensuring that the system fails gracefully rather than crashing on malformed output.

## Practical Implementation

### Consuming the State Generator

The primary interface for clients is the `statelessGenerate` async generator, which yields parsed events as they become available:

```typescript
import { statelessGenerate } from '@/lib/orchestration/stateless-generate';
import { createLogger } from '@/lib/logger';

const request = { /* StatelessChatRequest configuration */ };
const abort = new AbortController();

for await (const ev of statelessGenerate(
  request,
  abort.signal,
  languageModel,
)) {
  if (ev.type === 'text_delta') {
    // Render streamed teacher speech immediately
    console.log('🗨️', ev.data.content);
  } else if (ev.type === 'action') {
    // Queue a silent tool call for execution
    console.log('⚙️ Action:', ev.data.actionName, ev.data.params);
  } else if (ev.type === 'done') {
    console.log('✅ Generation finished', ev.data);
  }
}

```

This pattern streams events exactly as they are parsed from the interleaved JSON array, enabling real-time UI updates alongside background tool execution.

### Low-Level Parser API

For scenarios requiring direct control over the parsing process, the underlying state machine is exposed:

```typescript
import {
  createParserState,
  parseStructuredChunk,
  finalizeParser,
} from '@/lib/orchestration/stateless-generate';

const state = createParserState();

const chunks = [
  '`json\n[{"type":"text","content":"Hello ',
  'students!"},{"type":"action","name":"highlight","params":{"elementId":"img_1"}}',
  ']`',
];

for (const chunk of chunks) {
  const res = parseStructuredChunk(chunk, state);
  console.log('text:', res.textChunks);
  console.log('actions:', res.actions);
}

const final = finalizeParser(state);
console.log('final text:', final.textChunks);

```

This demonstrates the incremental parsing API: each call to `parseStructuredChunk` processes incoming data, emits newly available content, and preserves the original interleaving via the `ordered` array.

## Integration with the Orchestration Layer

The stateless generate parser integrates with the broader OpenMAIC architecture through [`lib/orchestration/director-graph.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/director-graph.ts), which orchestrates multi-agent conversation turns by importing the parser helpers to feed the LangGraph stream. Additionally, `packages/@openmaic/generation/src/action-parser.ts` bridges the stateless parser with the UI layer, exposing the same parsing utilities to frontend components. Together, these files implement the robust handling of interleaved structured JSON actions and freeform text across the entire OpenMAIC stack.

## Summary

- OpenMAIC expects a single JSON array containing alternating **action** and **text** objects, supporting both modern and legacy schema formats.
- The **`parseStructuredChunk`** function in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts) implements incremental parsing with prefix stripping (lines 50‑58) and dual JSON recovery using `jsonrepair` and `partial‑json` (lines 65‑78).
- Complete items are emitted via **`emitItem`** while preserving sequence through the **`ordered`** array (lines 75‑77).
- Partial text content streams incrementally using **`lastPartialTextLength`** tracking (lines 34‑44) for low-latency display.
- **`finalizeParser`** (lines 57‑78) recovers plain text from malformed output and suppresses unparseable fragments.
- The **`statelessGenerate`** async generator yields typed events that clients consume to render text and execute actions in the exact order specified by the LLM.

## Frequently Asked Questions

### What JSON format does OpenMAIC expect for interleaved actions and text?

OpenMAIC expects a single top-level JSON array where items alternate between two object types: `{"type":"action","name":"string","params":{}}` for silent tool calls and `{"type":"text","content":"string"}` for displayable content. The system also supports a legacy format using `tool_name` and `parameters` keys for backward compatibility with older model outputs.

### How does the parser handle incomplete or malformed JSON chunks?

The parser implements a two-stage recovery process at lines 65‑78 of [`stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/stateless-generate.ts). It first attempts to repair the JSON using the `jsonrepair` library, and if that fails, it falls back to `partial‑json` to extract whatever valid structure is available. This ensures the system can process streaming output even when the LLM produces syntax errors or the connection interrupts mid-object.

### What happens if the LLM output doesn't include a top-level JSON array?

When the LLM fails to produce a proper JSON array wrapper, the **`finalizeParser`** function (lines 57‑78) attempts to salvage any standalone `type:"text"` objects from the output buffer. It suppresses unparsable fragments to prevent crashes and logs the suppression for debugging, allowing the conversation to continue gracefully even when the model deviates from the expected format.

### How does the system maintain the order of mixed text and action items?

The parser maintains an **`ordered`** array within the `ParseResult` structure (lines 75‑77) that records every successfully parsed item in the exact sequence it appeared in the LLM output. This ensures that when `statelessGenerate` yields events to the client, text and action events arrive in the correct chronological order, preserving the pedagogical flow intended by the model.