What Kind of Data Does OmniRoute Stream? A Complete Guide to SSE Payloads

OmniRoute streams Server-Sent Events (SSE) containing JSON payloads that mirror AI provider APIs, including incremental chat tokens, tool-call signals, finish reasons, and usage statistics.

The diegosouzapw/OmniRoute proxy transforms requests to various AI providers into a unified streaming interface. Understanding what kind of data OmniRoute streams helps developers build real-time applications that consume LLM outputs, tool results, and completion signals as they arrive.

Understanding OmniRoute's SSE Payload Structure

OmniRoute transmits Server-Sent Events (SSE) with a Content-Type: text/event-stream header. Each event carries a JSON payload representing incremental generation output. The stream consists of discrete chunks that map to specific stages of the AI interaction.

Chat Token Chunks

The most common payload type contains individual tokens or text fragments from the language model. These chunks include fields such as id, object, created, model, and the actual content within choices[0].delta.content. As the LLM generates text, OmniRoute forwards these fragments immediately, allowing clients to display streaming output character-by-character.

Tool-Call Signals

When the model initiates tool usage, OmniRoute streams specific delta objects indicating the transition:

• Tool-call start: Contains choices[0].delta.tool_calls[0].id, name, and index fields signaling the beginning of a function invocation.
• Tool-call arguments: Streams partial JSON arguments via choices[0].delta.tool_calls[0].arguments as the model constructs the parameter object.
• Tool-call result: After execution, the tool's response inserts back into choices[0].delta.content, maintaining the conversation flow.

Completion Signals and Usage Statistics

The final chunks of a stream deliver metadata critical for application logic and billing:

• Finish reason: The choices[0].finish_reason field contains values like stop, length, tool_calls, or error, indicating why generation halted.
• Usage stats: Typically sent in the final chunk, usage.prompt_tokens, usage.completion_tokens, and usage.total_tokens provide token counts for quota management and cost tracking.

How OmniRoute Formats Streamed Data

All payloads conform to the SSE specification using the data: prefix syntax. A typical line appears as:

data: {"id":"chatcmpl-…","choices":[{"delta":{"content":"Hello"},"index":0}]}

The stream terminates with a line containing only data: [DONE]. If an error occurs during processing, the final line contains data: {"error":…} instead.

OmniRoute also implements keep-alive pings (event: keepalive) through the earlyStreamKeepalive utility. These prevent HTTP timeouts when the client requests streaming but the upstream model remains in a processing state, ensuring stable long-lived connections.

Endpoints Supporting Streaming

According to the source code in src/app/api/v1/chat/completions/route.ts, OmniRoute applies SSE formatting to multiple endpoints:

• Chat completions (/v1/chat/completions): The core handler checks for Accept: text/event-stream headers or stream: true parameters to activate SSE mode.
• Responses API (/v1/responses): Uses createResponsesApiTransformStream to convert chat-completion SSE into the OpenAI Responses API shape.
• Tool-heavy endpoints: Code execution and web-search results interleave with LLM output using the same delta format.
• Media generation: Audio, image, and video generation endpoints return SSE-style progress feeds when the upstream provider supports streaming.

Consuming OmniRoute Streams

Clients consume these streams using standard HTTP readers with SSE parsing logic.

Node.js or Browser Fetch Example

const resp = await fetch('http://localhost:20128/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Accept': 'text/event-stream',
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Explain streaming.' }],
    stream: true,
  }),
});

const reader = resp.body!.getReader();
const decoder = new TextDecoder();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  const chunk = decoder.decode(value);
  
  for (const line of chunk.split('\n')) {
    if (line.startsWith('data:')) {
      const payload = JSON.parse(line.slice(5).trim());
      console.log('Received:', payload);
    }
  }
}

Command-Line Testing

curl -N -H "Accept: text/event-stream" \
     -H "Content-Type: application/json" \
     -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Stream me tokens"}],"stream":true}' \
     http://localhost:20128/v1/chat/completions

The -N flag disables buffering, allowing you to observe each data: line as it arrives from the OmniRoute proxy.

Key Implementation Files

The streaming pipeline relies on specific modules within the repository:

• src/app/api/v1/chat/completions/route.ts: HTTP entry point that detects streaming requests via Accept headers and stream flags, delegating to the chat handler.
• src/sse/handlers/chat.ts: Core SSE generator that yields token-by-token JSON objects from upstream providers.
• open-sse/transformer/responsesTransformer.ts: Transforms chat-completion SSE format into the OpenAI Responses API structure.
• open-sse/utils/jsonToSse.ts: Utility that serializes JSON objects into SSE-compliant lines.
• open-sse/utils/earlyStreamKeepalive.ts: Injects periodic keep-alive events during upstream processing delays.
• open-sse/utils/aiSdkCompat.ts: Detects when SDKs force streaming through headers despite omitting stream: true.
• open-sse/utils/keepaliveThreshold.ts: Configures ping frequency based on specific model characteristics.

Summary

• OmniRoute streams JSON payloads wrapped in Server-Sent Events (SSE) syntax, delivering text/event-stream content to clients.
• The data includes incremental chat tokens, tool-call metadata, finish reasons, and usage statistics that mirror the underlying AI provider's API.
• Streams end with data: [DONE] or error payloads, with keep-alive pings maintaining connections during processing delays.
• The implementation spans src/app/api/v1/chat/completions/route.ts, src/sse/handlers/chat.ts, and various utilities in the open-sse directory.

Frequently Asked Questions

What format does OmniRoute use for streaming?

OmniRoute uses Server-Sent Events (SSE) with a Content-Type: text/event-stream header. Each line begins with data: followed by a JSON object representing incremental LLM output. This format is compatible with standard HTTP clients and matches the streaming interface of major AI providers like OpenAI.

How does OmniRoute handle tool calls in streams?

Tool calls appear as distinct delta types within the SSE stream. When a model initiates a tool, OmniRoute sends a chunk with choices[0].delta.tool_calls containing the tool ID and name. Arguments stream incrementally through the same field, and results merge back into the content stream after execution, allowing real-time observation of the tool-use lifecycle.

What is the keep-alive ping in OmniRoute?

The keep-alive ping is an SSE event (event: keepalive) injected by open-sse/utils/earlyStreamKeepalive.ts when the client requests streaming but the upstream model has not yet produced tokens. This prevents HTTP timeouts on long-running requests, ensuring the connection remains active during processing delays without requiring client-side reconnection logic.

How do I know when an OmniRoute stream ends?

The stream signals completion by sending a line containing exactly data: [DONE]. Prior to this, the final meaningful chunk typically includes choices[0].finish_reason set to stop, length, or tool_calls, along with usage statistics. If an error occurs, the final line contains a JSON error object instead of the [DONE] marker.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}