# How OpenMAIC Uses partial-json for Incremental Parsing of LLM Output

> Learn how OpenMAIC uses partial-json for incremental parsing of LLM output. Extract valid JSON from incomplete responses in real-time for efficient processing.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-03

---

**OpenMAIC leverages the `partial-json` library to extract valid JSON elements from incomplete, streaming LLM responses, enabling real-time processing of actions and text before the entire array is received.**

OpenMAIC is an open-source orchestration framework that generates structured output mixing executable actions and spoken text from large language models. Because these models emit tokens incrementally through streaming APIs, the resulting JSON array often arrives broken or incomplete mid-generation. The project solves this latency challenge by implementing **OpenMAIC partial-json incremental parsing** to recover usable objects from malformed buffers without waiting for the final closing bracket.

## Where OpenMAIC Implements partial-json Parsing

The framework employs `partial-json` in two distinct locations depending on whether the LLM response is being processed as a stream or as a complete payload.

### Offline Parsing in action-parser.ts

For non-streaming scenarios, OpenMAIC uses `packages/@openmaic/generation/src/action-parser.ts` to parse finalized LLM replies. The `parseActionsFromStructuredOutput` function attempts a strict `JSON.parse` first, falls back to `jsonrepair` for minor syntax errors, and finally invokes `partial-json` as a safety net when the array remains malformed (for example, missing a trailing `]`). This ensures that even badly formed complete responses yield actionable data.

### Streaming Incremental Parsing in stateless-generate.ts

Real-time generation happens in [`lib/orchestration/stateless-generate.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/orchestration/stateless-generate.ts). Here, the `parseStructuredChunk` function accumulates incoming chunks into a buffer and uses `partial-json` to evaluate which array elements have become syntactically complete. This allows the system to emit `textChunk` or `ParsedAction` objects immediately—preserving the original interleaving of speech and actions—rather than buffering until the stream terminates.

## How partial-json Handles Incomplete JSON Streams

The integration follows a specific pipeline to maximize fault tolerance and minimize latency.

### Importing and Configuring the Parser

OpenMAIC imports the permissive parser with specific flags to accept any value type:

```typescript
import { parse as parsePartialJson, Allow } from 'partial-json';

```

When parsing, it applies a bitwise mask instructing the library to tolerate incomplete arrays, objects, strings, numbers, booleans, and null values:

```typescript
parsePartialJson(
  jsonStr,
  Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL,
);

```

This configuration is critical because it allows the parser to succeed even when the JSON array is still open or contains partially formed elements.

### Detecting Array Boundaries

Before parsing, the streaming logic in `parseStructuredChunk` scans the accumulated buffer for the first `[` character, discarding any preceding content such as markdown fences (```` ```json ````). This isolates the actual JSON payload from LLM-generated formatting artifacts.

### Incremental Emission with completeUpTo

After each new chunk arrives, the parser re-evaluates the entire buffer. It determines the index (`completeUpTo`) up to which elements are fully formed. For a streaming array, the last item typically remains partial until additional tokens close its syntax; the parser ignores this trailing fragment and returns only the completed predecessors.

Completed items are immediately transformed into either:
- **textChunk**: When `type` equals `'text'`
- **ParsedAction**: When `type` equals `'action'`

The system records the original ordering, ensuring downstream graph logic maintains the exact sequence of speech and actions intended by the model.

### Fallback for Non-Streaming Cases

In `parseActionsFromStructuredOutput`, the same three-stage fallback sequence protects against malformed offline responses: strict `JSON.parse` → `jsonrepair` → `partial-json`. This guarantees data recovery even if the LLM omits structural characters like commas or closing brackets.

## Practical Code Examples

### Streaming Real-Time Parsing

The following example demonstrates how to process incremental chunks using the stateful parser implemented in `stateless-generate.ts`:

```typescript
import { createParserState, parseStructuredChunk } from '@/lib/orchestration/stateless-generate';

let state = createParserState();

// Simulate receiving chunks from the LLM
const chunks = [
  '```json\n[{"type":"action","name":"spotlight","params":{',
  '"elementId":"img_1"}},{"type":"text","content":"Hello',
  ' students!"}]\n```',
];

for (const chunk of chunks) {
  const result = parseStructuredChunk(chunk, state);
  console.log('New text chunks:', result.textChunks);
  console.log('New actions:', result.actions);
}

```

Each iteration immediately yields any completed actions or text segments without waiting for the final chunk.

### Parsing Complete Responses Offline

For batch processing of finalized LLM output, use the parser from `action-parser.ts`:

```typescript
import { parseActionsFromStructuredOutput } from '@/packages/@openmaic/generation/src/action-parser';

const llmReply = `
\`\`\`json
[
  {"type":"action","name":"spotlight","params":{"elementId":"img_1"}},
  {"type":"text","content":"Welcome to the session!"}
]
\`\`\`
`;

const actions = parseActionsFromStructuredOutput(llmReply);
console.log(actions);

```

This function automatically invokes `partial-json` if standard parsing fails, ensuring robust recovery from malformed arrays.

## Summary

- **OpenMAIC** integrates `partial-json` in `action-parser.ts` for offline recovery and `stateless-generate.ts` for streaming incremental parsing.
- The parser uses a permissive bitmask (`Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL`) to tolerate incomplete JSON structures mid-stream.
- The `completeUpTo` mechanism identifies fully formed array elements, enabling immediate emission of `textChunk` and `ParsedAction` objects while buffering partial trailing content.
- Fallback logic in `parseActionsFromStructuredOutput` ensures data extraction succeeds even when the LLM generates malformed or unclosed JSON arrays.

## Frequently Asked Questions

### What is partial-json and why does OpenMAIC need it?

`partial-json` is a JavaScript/TypeScript library designed to parse incomplete JSON strings without throwing syntax errors. OpenMAIC requires it because LLMs stream responses token-by-token, often producing arrays that are syntactically broken mid-generation (missing closing brackets, trailing commas, or unclosed strings). Without `partial-json`, the system would have to wait for the entire stream to finish before extracting any actions, introducing unacceptable latency in interactive applications.

### How does OpenMAIC distinguish between markdown fences and actual JSON?

In the streaming parser (`parseStructuredChunk`), the code explicitly searches for the first `[` character in the accumulated buffer and discards everything preceding it. This strips markdown code fences (such as ```` ```json ````) and other explanatory text the LLM might prepend, isolating the pure JSON array for `partial-json` processing.

### Can partial-json recover data if the LLM never closes the JSON array?

Yes. `partial-json` produces a best-effort parse up to the last syntactically valid point. According to the implementation in [`action-parser.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/action-parser.ts), even if the final `]` is never emitted, the parser returns all complete objects encountered before the break. The `jsonrepair` fallback also attempts to fix simple structural issues before `partial-json` is invoked, maximizing data recovery from aborted or truncated streams.

### Why does OpenMAIC use a bitwise mask when calling partial-json?

The bitmask (`Allow.ARR | Allow.OBJ | Allow.STR | Allow.NUM | Allow.BOOL | Allow.NULL`) configures the parser to accept any valid JSON value type while tolerating incompleteness. This is necessary because the LLM output mixes action objects and text strings within a single array, and individual elements may be partially received. The permissive flags prevent the parser from failing when encountering an incomplete string or an object missing its closing brace.