Handling Interleaved Structured JSON Actions and Freeform Text in OpenMAIC's Stateless Generate Process

OpenMAIC processes interleaved structured JSON actions and freeform text through a streaming JSON array parser that incrementally extracts complete items while preserving their original order and gracefully handling malformed output.

OpenMAIC's stateless generate pipeline enables large language models to emit mixed content types that combine silent tool executions with natural language responses. The system expects the LLM to output a single JSON array containing alternating objects that represent either executable actions or displayable text, processed in real-time through a robust streaming parser implemented in lib/orchestration/stateless-generate.ts.

The Interleaved JSON Array Format

The stateless generate process requires the LLM to structure its output as a JSON array containing two distinct object shapes that alternate throughout the stream:

• Action objects: {"type":"action","name":"…","params":{…}} (or the legacy {"tool_name":"…","parameters":{…}} format) represent silent tool calls that the client executes without displaying to the user.
• Text objects: {"type":"text","content":"…"} contain natural teacher speech meant for direct display.

This interleaved structure allows the model to switch between explaining concepts and triggering interface actions within a single generation stream.

Streaming Parser Implementation

The core parsing logic resides in the parseStructuredChunk function, which maintains state across streaming chunks to build a complete parse tree incrementally.

Buffer Management and Prefix Stripping

As LLM chunks arrive, the parser accumulates them in a buffer (lines 48‑59). The implementation specifically skips any prefix content before the opening [ character, such as markdown code fences or explanatory text, ensuring that parsing begins at the actual JSON array boundary (lines 50‑58).

Malformed JSON Recovery

To handle incomplete or malformed JSON common in streaming scenarios, the parser employs a dual-recovery strategy (lines 65‑78). It first attempts repair using jsonrepair, and if that fails, falls back to partial‑json. This robust approach allows the system to extract usable data even when the LLM produces malformed syntax or the stream is interrupted.

Deterministic Item Ordering

When items are successfully parsed, the emitItem function (lines 75‑77) records each new object in a ParseResult structure. Crucially, the parser maintains an ordered array that preserves the exact sequence of text and action objects as they appeared in the original LLM output, ensuring the client can render content and execute actions in the correct chronological order.

Processing Complete and Partial Items

The parser distinguishes between fully complete items and partially received content to enable low-latency streaming.

Emitting Fully Formed Items

After a successful JSON parse, the code determines which array items are fully complete using the completeUpTo index (lines 86‑101). Only complete items are emitted immediately, while incomplete fragments remain in the buffer awaiting additional chunks. This prevents the system from prematurely processing truncated actions or cut-off text.

Streaming Partial Text Deltas

For text items that are partially received but still incomplete, the parser implements delta streaming to minimize latency. It tracks the length of text already emitted via lastPartialTextLength (lines 34‑44) and streams only the newly available characters to the client. This approach allows the UI to display text in real-time without waiting for the entire JSON object to finish.

Detecting Stream Completion

When the parser encounters the closing ] character that terminates the JSON array, it sets state.isDone to true and marks the result as complete (lines 46‑52). This signals to the statelessGenerate async generator that the stream has finished and no further chunks are expected.

Error Recovery and Finalization

If the LLM output never produces a top-level JSON array, the finalizeParser function (lines 57‑78) attempts to recover any plain-text objects with type:"text" that it can extract from the debris. The function suppresses stray JSON fragments that cannot be parsed and logs these suppressions for debugging, ensuring that the system fails gracefully rather than crashing on malformed output.

Practical Implementation

Consuming the State Generator

The primary interface for clients is the statelessGenerate async generator, which yields parsed events as they become available:

import { statelessGenerate } from '@/lib/orchestration/stateless-generate';
import { createLogger } from '@/lib/logger';

const request = { /* StatelessChatRequest configuration */ };
const abort = new AbortController();

for await (const ev of statelessGenerate(
  request,
  abort.signal,
  languageModel,
)) {
  if (ev.type === 'text_delta') {
    // Render streamed teacher speech immediately
    console.log('🗨️', ev.data.content);
  } else if (ev.type === 'action') {
    // Queue a silent tool call for execution
    console.log('⚙️ Action:', ev.data.actionName, ev.data.params);
  } else if (ev.type === 'done') {
    console.log('✅ Generation finished', ev.data);
  }
}

This pattern streams events exactly as they are parsed from the interleaved JSON array, enabling real-time UI updates alongside background tool execution.

Low-Level Parser API

For scenarios requiring direct control over the parsing process, the underlying state machine is exposed:

import {
  createParserState,
  parseStructuredChunk,
  finalizeParser,
} from '@/lib/orchestration/stateless-generate';

const state = createParserState();

const chunks = [
  '`json\n[{"type":"text","content":"Hello ',
  'students!"},{"type":"action","name":"highlight","params":{"elementId":"img_1"}}',
  ']`',
];

for (const chunk of chunks) {
  const res = parseStructuredChunk(chunk, state);
  console.log('text:', res.textChunks);
  console.log('actions:', res.actions);
}

const final = finalizeParser(state);
console.log('final text:', final.textChunks);

This demonstrates the incremental parsing API: each call to parseStructuredChunk processes incoming data, emits newly available content, and preserves the original interleaving via the ordered array.

Integration with the Orchestration Layer

The stateless generate parser integrates with the broader OpenMAIC architecture through lib/orchestration/director-graph.ts, which orchestrates multi-agent conversation turns by importing the parser helpers to feed the LangGraph stream. Additionally, packages/@openmaic/generation/src/action-parser.ts bridges the stateless parser with the UI layer, exposing the same parsing utilities to frontend components. Together, these files implement the robust handling of interleaved structured JSON actions and freeform text across the entire OpenMAIC stack.

Summary

• OpenMAIC expects a single JSON array containing alternating action and text objects, supporting both modern and legacy schema formats.
• The parseStructuredChunk function in lib/orchestration/stateless-generate.ts implements incremental parsing with prefix stripping (lines 50‑58) and dual JSON recovery using jsonrepair and partial‑json (lines 65‑78).
• Complete items are emitted via emitItem while preserving sequence through the ordered array (lines 75‑77).
• Partial text content streams incrementally using lastPartialTextLength tracking (lines 34‑44) for low-latency display.
• finalizeParser (lines 57‑78) recovers plain text from malformed output and suppresses unparseable fragments.
• The statelessGenerate async generator yields typed events that clients consume to render text and execute actions in the exact order specified by the LLM.

Frequently Asked Questions

What JSON format does OpenMAIC expect for interleaved actions and text?

OpenMAIC expects a single top-level JSON array where items alternate between two object types: {"type":"action","name":"string","params":{}} for silent tool calls and {"type":"text","content":"string"} for displayable content. The system also supports a legacy format using tool_name and parameters keys for backward compatibility with older model outputs.

How does the parser handle incomplete or malformed JSON chunks?

The parser implements a two-stage recovery process at lines 65‑78 of stateless-generate.ts. It first attempts to repair the JSON using the jsonrepair library, and if that fails, it falls back to partial‑json to extract whatever valid structure is available. This ensures the system can process streaming output even when the LLM produces syntax errors or the connection interrupts mid-object.

What happens if the LLM output doesn't include a top-level JSON array?

When the LLM fails to produce a proper JSON array wrapper, the finalizeParser function (lines 57‑78) attempts to salvage any standalone type:"text" objects from the output buffer. It suppresses unparsable fragments to prevent crashes and logs the suppression for debugging, allowing the conversation to continue gracefully even when the model deviates from the expected format.

How does the system maintain the order of mixed text and action items?

The parser maintains an ordered array within the ParseResult structure (lines 75‑77) that records every successfully parsed item in the exact sequence it appeared in the LLM output. This ensures that when statelessGenerate yields events to the client, text and action events arrive in the correct chronological order, preserving the pedagogical flow intended by the model.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}