# How to Set Up OmniRoute for Streaming: Server-Sent Events Configuration

> Learn how to set up OmniRoute for streaming with Server-Sent Events. Configure requests for real-time AI outputs using SSE and streamlined pipelines.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-02

---

**OmniRoute delivers AI-generated outputs as Server-Sent Events (SSE) when clients set `stream: true` in the JSON body or send an `Accept: text/event-stream` header, orchestrated through the request handler in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) and the streaming pipeline in [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts).**

OmniRoute is an open-source AI gateway that provides OpenAI-compatible streaming endpoints. When you set up OmniRoute for streaming, the system detects streaming intent via request parameters or headers, then pipes provider responses through a specialized pipeline that maintains low-latency delivery to downstream clients.

## Understanding OmniRoute Streaming Architecture

The streaming implementation follows a four-stage pipeline that transforms upstream provider responses into compliant SSE streams.

### Request Detection Logic

The entry point at [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) evaluates the `wantsStreaming` boolean by checking two conditions:

```typescript
const wantsStreaming =
  (parsedBodyIsRecord && parsedBody.stream === true) || // explicit true
  acceptForcesStream;                                   // Accept header forces SSE

```

The `acceptForcesStream` variable, implemented in [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts), treats `Accept: text/event-stream` as an implicit `stream: true` when the request body omits the flag.

### Core Pipeline Assembly

The [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts) file contains the streaming orchestration logic. It implements `resolveStreamFlag` to determine the final streaming mode and executes `assembleStreamingPipeline` to construct the processing chain. This handler manages keep-alive pings and translates provider-specific errors into standardized SSE events.

### Provider Execution Layer

Provider-specific executors in the `open-sse/executors/` directory (such as [`default.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/default.ts)) forward requests to upstream APIs with `stream: true` enabled. These executors return a `ReadableStream` from the provider's HTTP response, which preserves the byte-stream nature of the data without buffering.

### Stream Finalization

The [`src/app/api/v1/relay/chat/completions/streamFinalizer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/streamFinalizer.ts) file bridges the upstream `ReadableStream` to the HTTP response. Its `finalizeReadableStream` function converts raw provider bytes into properly formatted SSE messages:

```typescript
const stream = finalizeReadableStream(upstream.body, (error) => { … });
return new Response(stream, { status: upstream.status, headers });

```

## Environment Configuration Variables

Several environment variables control streaming behavior and safety limits:

| Variable | Default | Effect |
|----------|---------|--------|
| `OMNIROUTE_MAX_NONSTREAMING_RESPONSE_BYTES` | `67108864` (64 MiB) | Hard-cap for buffered non-streaming responses before rejection |
| `STREAM_READINESS_MAX_TIMEOUT_MS` | `180000` ms | Maximum wait time for the first upstream token before the stream is considered ready |
| `BIFROST_STREAMING_ENABLED` | `true` | Enables streaming on the optional Bifrost sidecar route |
| `OMNIROUTE_AGENT_GOAL_STREAM_RECOVERY` | `true` | Adds early-stream recovery mechanisms for special `/goal` workflows |
| `OMNIROUTE_STREAMING_DEFAULT` | `true` | Global default for streaming; set to `"false"` to force JSON responses even when clients request SSE |

Configure these in your `.env` file after cloning the repository.

## Step-by-Step Setup Guide

Follow these steps to configure OmniRoute for streaming responses.

### 1. Install Dependencies

```bash
git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
npm ci

```

### 2. Configure Environment Variables

Create a `.env` file from the example template. At minimum, provide an API key for your target provider:

```dotenv
OPENAI_API_KEY=sk-...

```

To disable streaming globally and force JSON responses regardless of client headers:

```dotenv
OMNIROUTE_STREAMING_DEFAULT=false

```

### 3. Start the Server

```bash
npm run dev

```

The server listens on `http://localhost:3000` by default.

### 4. Make Streaming Requests

**Using the `stream` flag:**

```bash
curl http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Tell me a joke." }],
    "stream": true
  }'

```

**Using the `Accept` header only:**

```bash
curl http://localhost:3000/v1/chat/completions \
  -H "Accept: text/event-stream" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Write a haiku." }]
  }'

```

Both methods return SSE events formatted as:

```

data: {"id":"…","choices":[{"delta":{"content":"..."},"index":0,"finish_reason":null}]}

```

### 5. Configure the Bifrost Sidecar (Optional)

The Bifrost sidecar proxies downstream providers while preserving streaming semantics. Enable it by setting:

```dotenv
BIFROST_API_KEY=sk-...
BIFROST_BASE_URL=https://api.provider.com
BIFROST_STREAMING_ENABLED=true

```

Request streaming via the sidecar route:

```bash
curl http://localhost:3000/v1/relay/chat/completions/bifrost \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Explain quantum tunneling."}],"stream":true}'

```

The Bifrost handler in [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/bifrost/route.ts) evaluates the `wantsStream` boolean using both the request body flag and the `BIFROST_STREAMING_ENABLED` environment variable.

## Client Implementation Examples

### Node.js Streaming Client

```javascript
import fetch from "node-fetch";

async function streamChat() {
  const response = await fetch("http://localhost:3000/v1/chat/completions", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: "List three fruits." }],
      stream: true,
    }),
  });

  if (!response.body) throw new Error("No stream available");

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    process.stdout.write(decoder.decode(value));
  }
}

streamChat().catch(console.error);

```

### Command-Line with Buffering Disabled

```bash
curl -N http://localhost:3000/v1/chat/completions \
  -H "Accept: text/event-stream" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Summarize the plot of Hamlet"}]}'

```

The `-N` flag disables curl's output buffering, allowing you to see SSE chunks as they arrive from the server.

## Key Source Files Reference

| Component | File Path | Purpose |
|-----------|-----------|---------|
| **API Entry** | [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) | Detects streaming intent via `wantsStreaming` and routes to core handlers |
| **Core Handler** | [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts) | Implements `resolveStreamFlag` and `assembleStreamingPipeline` for stream orchestration |
| **Stream Finalizer** | [`src/app/api/v1/relay/chat/completions/streamFinalizer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/streamFinalizer.ts) | Converts `ReadableStream` to SSE responses via `finalizeReadableStream` |
| **Bifrost Route** | [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/bifrost/route.ts) | Sidecar proxy that respects `BIFROST_STREAMING_ENABLED` |
| **Transform Utilities** | [`open-sse/transformer/stream.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/transformer/stream.ts) | Low-level `TransformStream` that normalizes provider events to OpenAI-compatible chunks |
| **Configuration** | [`docs/reference/ENVIRONMENT.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/ENVIRONMENT.md) | Documents streaming-related environment variables |

## Summary

- **OmniRoute** streams responses via **Server-Sent Events (SSE)** when it detects `stream: true` in the request body or `Accept: text/event-stream` in headers.
- The **streaming pipeline** runs through [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) → [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts) → executor → [`streamFinalizer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/streamFinalizer.ts).
- **Environment variables** like `STREAM_READINESS_MAX_TIMEOUT_MS` and `BIFROST_STREAMING_ENABLED` fine-tune timeout behavior and sidecar capabilities.
- The **Bifrost sidecar** in [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/bifrost/route.ts) provides additional proxy functionality while maintaining streaming semantics.
- No additional code changes are required to enable streaming; simply configure your API keys and set the appropriate request flags.

## Frequently Asked Questions

### How does OmniRoute detect that a client wants streaming?

OmniRoute evaluates the `wantsStreaming` boolean in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts), which returns true if the JSON body contains `"stream": true` **or** if the `Accept` header equals `text/event-stream`. The latter triggers the `acceptForcesStream` logic in [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts), allowing header-only streaming negotiation.

### Can I disable streaming globally for all clients?

Yes. Set `OMNIROUTE_STREAMING_DEFAULT=false` in your environment configuration. This forces JSON responses even when clients send `stream: true` or `Accept: text/event-stream`, effectively overriding client preferences for scenarios where buffering is required.

### What is the purpose of the Bifrost sidecar in streaming?

The Bifrost sidecar acts as a reverse-proxy for downstream providers, exposed at `/v1/relay/chat/completions/bifrost`. When `BIFROST_STREAMING_ENABLED` is true, it passes the upstream `ReadableStream` through `finalizeReadableStream` to preserve SSE formatting, allowing you to chain providers while maintaining real-time output delivery.

### How does OmniRoute handle streaming timeouts?

The system uses `STREAM_READINESS_MAX_TIMEOUT_MS` (default 180,000 ms) to define the maximum wait time for the first meaningful token from the upstream provider. If the provider fails to send data within this window, the stream is considered failed and appropriate error handling is triggered according to the pipeline configuration in [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts).