How to Handle Streaming Tool Calls with Partial JSON Arguments in pi-ai

Accumulate partial JSON fragments in a buffer, parse them safely using parseStreamingJson on every delta, and emit toolcall_delta events for live UI updates until the final toolcall_end event completes the tool call.

When building AI applications with the badlogic/pi-mono repository, handling streaming tool calls requires special care because large language models (LLMs) emit function arguments as partial JSON chunks rather than complete objects. The pi-ai package implements a robust pipeline to accumulate these fragments, parse them safely without throwing errors, and propagate incremental updates to the UI layer.

Understanding the Streaming Tool Call Architecture in pi-ai

The streaming architecture in pi-ai separates concerns across three distinct layers: provider adapters for vendor-specific event translation, JSON utilities for safe parsing, and UI components for reactive rendering.

Provider Adapters and Event Standardization

Provider adapters in files like packages/ai/src/providers/openai-responses-shared.ts convert raw vendor events into standardized AssistantMessageEvent objects. When OpenAI emits a response.function_call_arguments.delta event, the adapter appends the delta to an internal partialJson buffer stored on the current tool-call block.

The adapter then immediately invokes parseStreamingJson to attempt parsing the accumulated buffer, emitting a toolcall_delta event containing both the raw delta and the partially parsed object.

The JSON Parsing Pipeline

The parsing pipeline relies on packages/ai/src/utils/json-parse.ts to handle malformed or incomplete JSON without throwing exceptions. This utility first attempts standard JSON.parse, and if that fails— which is expected during streaming—it falls back to the partial-json library to extract whatever object structure is available from the truncated input.

Accumulating and Parsing Partial JSON Fragments

Handling partial JSON requires maintaining state across multiple streaming events and safely parsing data that may be syntactically incomplete.

The Accumulation Buffer

Each active tool call maintains a partialJson string property that accumulates raw JSON fragments as they arrive from the LLM. In openai-responses-shared.ts, the provider adapter checks if the current block type is toolCall and appends the incoming delta:

if (event.type === "response.function_call_arguments.delta") {
    if (currentItem?.type === "function_call" && currentBlock?.type === "toolCall") {
        currentBlock.partialJson += event.delta;
        currentBlock.arguments = parseStreamingJson(currentBlock.partialJson);
        // Emit toolcall_delta event...
    }
}

Safe Parsing with parseStreamingJson

The parseStreamingJson function in packages/ai/src/utils/json-parse.ts implements a three-tier fallback strategy to ensure the UI never receives a parsing error:

export function parseStreamingJson<T = any>(partialJson: string | undefined): T {
    if (!partialJson?.trim()) return {} as T;
    try {
        return JSON.parse(partialJson) as T;
    } catch {
        try {
            const result = partialParse(partialJson);
            return (result ?? {}) as T;
        } catch {
            return {} as T;
        }
    }
}

This approach prioritizes speed for complete JSON while gracefully handling truncated objects through the partial-json library's partialParse function.

Finalizing the Tool Call

When the provider receives the completion signal—response.function_call_arguments.done for OpenAI—it performs a final parse and emits the toolcall_end event:

} else if (event.type === "response.function_call_arguments.done") {
    if (currentItem?.type === "function_call" && currentBlock?.type === "toolCall") {
        currentBlock.partialJson = event.arguments;
        currentBlock.arguments = parseStreamingJson(currentBlock.partialJson);
        // Emit toolcall_end event with complete ToolCall object
    }
}

At this stage, optional validation against TypeBox schemas in packages/ai/src/utils/validation.ts can verify the final arguments before execution.

Handling Streaming Tool Calls in the UI Layer

The web UI components consume the streaming events through AgentSession and must handle reactive updates to nested properties.

Reacting to toolcall_delta Events

The UI receives toolcall_delta events containing the incremental changes to tool arguments. These events allow real-time rendering of partial JSON as it streams from the model, enabling users to see the function parameters being constructed live.

Deep Cloning for Lit-Element Reactivity

In packages/web-ui/src/components/StreamingMessageContainer.ts, the component schedules updates using requestAnimationFrame and performs deep cloning to ensure Lit-Element detects mutations in nested arguments objects:

if (!this._updateScheduled) {
    this._updateScheduled = true;
    requestAnimationFrame(async () => {
        if (!this._immediateUpdate && this._pendingMessage !== null) {
            // Deep clone so Lit detects nested changes (e.g. arguments mutated)
            this._message = JSON.parse(JSON.stringify(this._pendingMessage));
            this.requestUpdate();
        }
        this._pendingMessage = null;
        this._updateScheduled = false;
        this._immediateUpdate = false;
    });
}

This pattern is essential because the provider adapters mutate the toolCall.arguments object in-place during streaming, and Lit-Element's change detection requires new object references to trigger re-renders.

Complete Implementation Example

The following example demonstrates setting up a streaming session with tool support, handling partial arguments, and executing the final tool call:

import { streamSimple } from "@mariozechner/pi-ai";

// 1️⃣ Start a streaming session with tool definitions
const session = await streamSimple("gpt-4o-mini", [
  { role: "user", content: "Create a file named data.json with {\"name\": \"Ada\"}" }
], {
  tools: [{ 
    name: "writeFile", 
    parameters: { 
      type: "object", 
      properties: { 
        path: {type:"string"}, 
        content:{type:"string"} 
      }, 
      required:["path","content"] 
    } 
  }],
});

// 2️⃣ Listen for incremental tool-call events
session.on("toolcall_delta", (ev) => {
  console.log("Partial args so far:", ev.delta);
  // UI can display ev.partial or the whole session.message with updated arguments
});

// 3️⃣ When the tool call finishes, validate and execute
session.on("toolcall_end", async (ev) => {
  const args = ev.toolCall.arguments;           // already fully parsed
  
  // Optional: strict validation against TypeBox schema
  const safeArgs = validateToolCall(tools, ev.toolCall); // from validation.ts
  
  await writeFile(safeArgs.path, safeArgs.content);
});

The toolcall_delta listener receives each JSON fragment as the LLM streams it, while toolcall_end provides the complete, parsed arguments ready for validation and execution.

Summary

• Accumulate partial JSON in a partialJson buffer on the active tool-call block as each delta arrives from the provider.
• Parse safely using parseStreamingJson in packages/ai/src/utils/json-parse.ts, which falls back to the partial-json library and never throws.
• Emit incremental events via toolcall_delta so UI components can render live argument construction.
• Finalize on completion when the provider signals the end of the stream, replacing the buffer with final arguments and emitting toolcall_end.
• Deep clone in the UI using JSON.parse(JSON.stringify(...)) in StreamingMessageContainer.ts to ensure Lit-Element detects nested mutations in toolCall.arguments.

Frequently Asked Questions

What happens if the LLM sends malformed JSON during streaming?

The parseStreamingJson function in packages/ai/src/utils/json-parse.ts handles malformed fragments gracefully. It first attempts standard JSON.parse, and if that fails, it falls back to the partial-json library's partialParse function. If all parsing attempts fail, it returns an empty object rather than throwing an error, ensuring the UI remains stable even with invalid partial input.

How does the UI know when to update during a streaming tool call?

The UI receives toolcall_delta events through the AgentSession event emitter. In packages/web-ui/src/components/StreamingMessageContainer.ts, the component schedules an update on the next animation frame using requestAnimationFrame. It performs a deep clone of the pending message using JSON.parse(JSON.stringify(...)) to create new object references, which triggers Lit-Element's change detection and causes a re-render with the updated arguments.

Can I validate the tool arguments before executing the final tool call?

Yes, after receiving the toolcall_end event, you can validate the final arguments against the tool's TypeBox schema using the utilities in packages/ai/src/utils/validation.ts. The validateToolCall function takes the tool definitions and the completed ToolCall object, returning validated and typed arguments that match the expected schema before you execute the actual function implementation.

What is the difference between toolcall_delta and toolcall_end events?

The toolcall_delta event fires every time the LLM sends a new JSON fragment during streaming, containing the incremental change and the current partial arguments. This allows real-time UI updates as the JSON object builds. The toolcall_end event fires once when the LLM signals completion, containing the final fully-parsed ToolCall object with complete arguments, indicating that no more deltas will arrive and the tool is ready for execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}