How OpenMAIC Uses partial-json for Incremental Parsing of LLM Output

OpenMAIC leverages the partial-json library to extract valid JSON elements from incomplete, streaming LLM responses, enabling real-time processing of actions and text before the entire array is received.

OpenMAIC is an open-source orchestration framework that generates structured output mixing executable actions and spoken text from large language models. Because these models emit tokens incrementally through streaming APIs, the resulting JSON array often arrives broken or incomplete mid-generation. The project solves this latency challenge by implementing OpenMAIC partial-json incremental parsing to recover usable objects from malformed buffers without waiting for the final closing bracket.

Where OpenMAIC Implements partial-json Parsing

The framework employs partial-json in two distinct locations depending on whether the LLM response is being processed as a stream or as a complete payload.

Offline Parsing in action-parser.ts

For non-streaming scenarios, OpenMAIC uses packages/@openmaic/generation/src/action-parser.ts to parse finalized LLM replies. The parseActionsFromStructuredOutput function attempts a strict JSON.parse first, falls back to jsonrepair for minor syntax errors, and finally invokes partial-json as a safety net when the array remains malformed (for example, missing a trailing ]). This ensures that even badly formed complete responses yield actionable data.

Streaming Incremental Parsing in stateless-generate.ts

Real-time generation happens in lib/orchestration/stateless-generate.ts. Here, the parseStructuredChunk function accumulates incoming chunks into a buffer and uses partial-json to evaluate which array elements have become syntactically complete. This allows the system to emit textChunk or ParsedAction objects immediately—preserving the original interleaving of speech and actions—rather than buffering until the stream terminates.

How partial-json Handles Incomplete JSON Streams

The integration follows a specific pipeline to maximize fault tolerance and minimize latency.

Importing and Configuring the Parser

OpenMAIC imports the permissive parser with specific flags to accept any value type:

import { parse as parsePartialJson, Allow } from 'partial-json';

When parsing, it applies a bitwise mask instructing the library to tolerate incomplete arrays, objects, strings, numbers, booleans, and null values:

parsePartialJson(
  jsonStr,
  Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL,
);

This configuration is critical because it allows the parser to succeed even when the JSON array is still open or contains partially formed elements.

Detecting Array Boundaries

Before parsing, the streaming logic in parseStructuredChunk scans the accumulated buffer for the first [ character, discarding any preceding content such as markdown fences (```json). This isolates the actual JSON payload from LLM-generated formatting artifacts.

Incremental Emission with completeUpTo

After each new chunk arrives, the parser re-evaluates the entire buffer. It determines the index (completeUpTo) up to which elements are fully formed. For a streaming array, the last item typically remains partial until additional tokens close its syntax; the parser ignores this trailing fragment and returns only the completed predecessors.

Completed items are immediately transformed into either:

• textChunk: When type equals 'text'
• ParsedAction: When type equals 'action'

The system records the original ordering, ensuring downstream graph logic maintains the exact sequence of speech and actions intended by the model.

Fallback for Non-Streaming Cases

In parseActionsFromStructuredOutput, the same three-stage fallback sequence protects against malformed offline responses: strict JSON.parse → jsonrepair → partial-json. This guarantees data recovery even if the LLM omits structural characters like commas or closing brackets.

Practical Code Examples

Streaming Real-Time Parsing

The following example demonstrates how to process incremental chunks using the stateful parser implemented in stateless-generate.ts:

import { createParserState, parseStructuredChunk } from '@/lib/orchestration/stateless-generate';

let state = createParserState();

// Simulate receiving chunks from the LLM
const chunks = [
  '```json\n[{"type":"action","name":"spotlight","params":{',
  '"elementId":"img_1"}},{"type":"text","content":"Hello',
  ' students!"}]\n```',
];

for (const chunk of chunks) {
  const result = parseStructuredChunk(chunk, state);
  console.log('New text chunks:', result.textChunks);
  console.log('New actions:', result.actions);
}

Each iteration immediately yields any completed actions or text segments without waiting for the final chunk.

Parsing Complete Responses Offline

For batch processing of finalized LLM output, use the parser from action-parser.ts:

import { parseActionsFromStructuredOutput } from '@/packages/@openmaic/generation/src/action-parser';

const llmReply = `
\`\`\`json
[
  {"type":"action","name":"spotlight","params":{"elementId":"img_1"}},
  {"type":"text","content":"Welcome to the session!"}
]
\`\`\`
`;

const actions = parseActionsFromStructuredOutput(llmReply);
console.log(actions);

This function automatically invokes partial-json if standard parsing fails, ensuring robust recovery from malformed arrays.

Summary

• OpenMAIC integrates partial-json in action-parser.ts for offline recovery and stateless-generate.ts for streaming incremental parsing.
• The parser uses a permissive bitmask (Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL) to tolerate incomplete JSON structures mid-stream.
• The completeUpTo mechanism identifies fully formed array elements, enabling immediate emission of textChunk and ParsedAction objects while buffering partial trailing content.
• Fallback logic in parseActionsFromStructuredOutput ensures data extraction succeeds even when the LLM generates malformed or unclosed JSON arrays.

Frequently Asked Questions

What is partial-json and why does OpenMAIC need it?

partial-json is a JavaScript/TypeScript library designed to parse incomplete JSON strings without throwing syntax errors. OpenMAIC requires it because LLMs stream responses token-by-token, often producing arrays that are syntactically broken mid-generation (missing closing brackets, trailing commas, or unclosed strings). Without partial-json, the system would have to wait for the entire stream to finish before extracting any actions, introducing unacceptable latency in interactive applications.

How does OpenMAIC distinguish between markdown fences and actual JSON?

In the streaming parser (parseStructuredChunk), the code explicitly searches for the first [ character in the accumulated buffer and discards everything preceding it. This strips markdown code fences (such as ```json) and other explanatory text the LLM might prepend, isolating the pure JSON array for partial-json processing.

Can partial-json recover data if the LLM never closes the JSON array?

Yes. partial-json produces a best-effort parse up to the last syntactically valid point. According to the implementation in action-parser.ts, even if the final ] is never emitted, the parser returns all complete objects encountered before the break. The jsonrepair fallback also attempts to fix simple structural issues before partial-json is invoked, maximizing data recovery from aborted or truncated streams.

Why does OpenMAIC use a bitwise mask when calling partial-json?

The bitmask (Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL) configures the parser to accept any valid JSON value type while tolerating incompleteness. This is necessary because the LLM output mixes action objects and text strings within a single array, and individual elements may be partially received. The permissive flags prevent the parser from failing when encountering an incomplete string or an object missing its closing brace.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}