How Tambo AI Handles Real-Time Streaming of Component Props from LLMs

Tambo AI streams component properties from LLMs as RFC-6902 JSON-Patch deltas, parsing partial JSON tokens in the backend and propagating typed events through React contexts to update UI components in real time.

Tambo AI enables generative interfaces by allowing Large Language Models (LLMs) to render React components dynamically. A critical challenge in this architecture is handling real-time streaming of component props from LLMs, where properties arrive as fragmented JSON tokens rather than complete objects. The tambo-ai/tambo repository solves this through a sophisticated pipeline that transforms raw LLM deltas into structured, incremental UI updates.

The Streaming Pipeline Architecture

The real-time streaming of component props from LLMs follows a six-stage pipeline:

1. LLM Tool Invocation – The model emits a tool call named show_component_<ComponentName> containing partial JSON payloads.
2. Backend Delta Processing – packages/backend/src/util/component-streaming.ts parses incoming strings using partial-json, detects changes, and generates JSON-Patch operations.
3. Event Emission – The backend emits ComponentStartEvent, ComponentPropsDeltaEvent, and ComponentEndEvent with per-prop streaming status.
4. SDK Stream Handling – react-sdk/src/v1/utils/stream-handler.ts forwards the async iterable to the React layer.
5. Context Accumulation – react-sdk/src/v1/providers/tambo-v1-stream-context.tsx reduces events into a normalized StreamState tree.
6. Hook Synchronization – react-sdk/src/v1/hooks/use-tambo-v1-component-state.ts applies patches to local component state and manages bidirectional sync.

Backend Processing: From LLM Deltas to JSON-Patch Events

The ComponentStreamTracker Class

At the heart of the backend processing lies the ComponentStreamTracker class defined in packages/backend/src/util/component-streaming.ts (lines 75-120). This class safeguards the streaming process by:

• Parsing partial JSON tokens using the partial-json library to handle malformed or incomplete JSON gracefully.
• Comparing immutable snapshots to detect added, changed, or removed properties without deep cloning overhead.
• Enforcing payload size limits to prevent memory exhaustion from runaway LLM outputs.
• Generating RFC-6902 JSON-Patch operations (add, replace, remove) that represent the minimal diff between states.

Event Types and Streaming Status

The tracker emits three distinct event types defined in react-sdk/src/v1/types/event.ts:

• ComponentStartEvent – Signals the beginning of a component stream, initializing the component instance in the client state tree.
• ComponentPropsDeltaEvent – Carries an array of JSON-Patch operations and a streamingStatus map indicating each prop's current phase: started, streaming, or done.
• ComponentEndEvent – Finalizes the component, marking all properties as done and completing the stream.

This granular status tracking allows the UI to render placeholder states for props still being generated while freezing interaction on finalized values.

SDK and React Integration

Stream Handler and Context Providers

The react-sdk/src/v1/utils/stream-handler.ts file acts as a thin bridge, forwarding the async iterable from client.threads.runs.run() into the React ecosystem with optional debug logging.

The react-sdk/src/v1/providers/tambo-v1-stream-context.tsx implements a split-context pattern to optimize rendering performance:

• StreamStateContext – Provides read-only access to the accumulated stream state.
• StreamDispatchContext – Exposes the dispatch function for injecting events.
• ThreadManagementContext – Handles thread lifecycle operations separately.

This separation prevents components that only need dispatch capabilities from re-rendering when state changes occur.

Event Accumulation and State Management

The streamReducer implemented in react-sdk/src/v1/utils/event-accumulator.ts processes incoming events into a normalized StreamState tree. This reducer:

• Maintains a mirror of the component hierarchy as defined by the LLM tool calls.
• Stores the latest propsDelta patches for each component instance.
• Handles out-of-order events gracefully through immutable updates.

The useTamboComponentState Hook

Synchronizing Props with the Backend

The react-sdk/src/v1/hooks/use-tambo-v1-component-state.ts file (definition starts at line 70) exposes a useState-like API that bridges the streaming backend with local React state:

const [value, setValue, { isPending, flush }] = useTamboComponentState<string>("title", "");

The hook performs three critical functions:

1. Reading – Retrieves current props from the StreamState tree using findComponentContent.
2. Patching – Applies incoming ComponentPropsDeltaEvent patches to local state as they arrive.
3. Writing – Debounces outgoing changes back to the backend via client.threads.state.updateState to reduce network chatter while maintaining eventual consistency.

Handling User Interactions

The hook distinguishes between rendered components (server-driven UI) and interactable components (client-side tools):

• For rendered components, state changes sync bidirectionally with the backend.
• For interactable components, changes forward to useTamboInteractable instead, keeping client-side tool state local until submission.

Code Examples

Consuming Streamed Props in React

import { useTamboComponentState } from "@tambo-ai/react";

export function ProductCard() {
  // The key must match the prop name emitted by the LLM
  const [price, setPrice, { isPending }] = useTamboComponentState<number>(
    "price", 
    0
  );

  return (
    <div className="product-card">
      <span className={isPending ? "streaming" : "final"}>
        ${price.toFixed(2)}
      </span>
      <button onClick={() => setPrice(p => p + 1)}>
        Increase Price
      </button>
    </div>
  );
}

Manually Flushing Pending Updates

When navigating away or submitting forms, ensure queued state reaches the server:

const [, , { flush }] = useTamboComponentState("description", "");

useEffect(() => {
  return () => {
    // Synchronous cleanup to guarantee delivery
    flush();
  };
}, [flush]);

Low-Level Event Handling

For debugging or synthetic event injection:

import { useStreamDispatch } from "@tambo-ai/react";

function StreamDebugger() {
  const dispatch = useStreamDispatch();

  const simulateUpdate = () => {
    dispatch({
      type: "EVENT",
      event: {
        type: "ComponentPropsDeltaEvent",
        componentId: "header-123",
        operations: [
          { op: "replace", path: "/title", value: "Updated Title" }
        ],
        streamingStatus: { title: "streaming" }
      }
    });
  };

  return <button onClick={simulateUpdate}>Inject Delta</button>;
}

Key Architectural Decisions

Decision	Implementation	Benefit
JSON-Patch (RFC-6902)	ComponentStreamTracker generates add, replace, remove operations	Minimizes payload size by transmitting only changed fields rather than full objects
Per-Property Streaming Status	ComponentPropsDeltaEvent includes streamingStatus map with started/streaming/done states	Enables UI placeholders and prevents interaction with incomplete data
Immutable Snapshots	partial-json library returns fresh objects; tracker compares references	Eliminates deep cloning overhead while ensuring accurate diff detection
Split-Context Pattern	TamboStreamProvider separates StreamStateContext, StreamDispatchContext, and ThreadManagementContext	Prevents re-renders in components that only need dispatch capabilities
Debounced Bidirectional Sync	useTamboComponentState batches outgoing changes via client.threads.state.updateState	Reduces network chatter while maintaining eventual consistency with the backend

Summary

Tambo AI achieves real-time streaming of component props from LLMs through a sophisticated pipeline that bridges backend JSON parsing with React state management:

• The backend uses ComponentStreamTracker in packages/backend/src/util/component-streaming.ts to parse partial JSON deltas and emit RFC-6902 JSON-Patch events.
• The React SDK consumes these events through useTamboComponentState, applying patches to local component state while debouncing user changes back to the server.
• Per-property streaming status (started, streaming, done) enables UIs to render progressive loading states and prevent premature interactions.

This architecture ensures that LLM-generated interfaces feel responsive and native, with properties appearing incrementally as the model generates them.

Frequently Asked Questions

What format does Tambo AI use for streaming component props?

Tambo AI uses RFC-6902 JSON-Patch operations to transmit property changes. Rather than sending complete JSON objects on every update, the backend emits compact patch documents containing add, replace, or remove operations. This format minimizes network payload size and aligns with standard delta-encoding practices.

How does the React SDK know when a prop has finished streaming?

Each ComponentPropsDeltaEvent includes a streamingStatus map that tracks individual properties through three phases: started (first byte received), streaming (delta active), and done (final value confirmed). The useTamboComponentState hook exposes this status via the isPending flag, allowing components to render loading indicators while properties remain in flux.

Can users interact with components while props are still streaming?

Yes, but with safeguards. The useTamboComponentState hook distinguishes between rendered components (server-driven) and interactable components (client-side tools). For rendered components, user changes are debounced and synced back to the backend via client.threads.state.updateState. This ensures the UI remains responsive while maintaining eventual consistency with the streaming LLM output.

What happens if the LLM generates invalid JSON during streaming?

The backend employs the partial-json library to parse incoming deltas, which gracefully handles malformed or incomplete JSON fragments. The ComponentStreamTracker class (lines 75-120 in packages/backend/src/util/component-streaming.ts) safeguards against oversized payloads and maintains immutable snapshots for accurate diff detection. If the JSON remains unparseable, the tracker simply waits for subsequent deltas to complete the structure before emitting events.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}