# OmniRoute Request Flow: How Open-SSE Handlers Route Requests to Executors

> Understand the OmniRoute request flow from Next.js API to Open-SSE handlers and executors. Learn how requests are routed through guards and transformation layers for efficient LLM calls.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-08-04

---

**OmniRoute processes chat completion requests through a multi-stage pipeline in the `open-sse` workspace, starting from the Next.js API route, passing through admission guards and transformation layers, before handing off to provider-specific executors that handle upstream LLM calls.**

The `open-sse` package in OmniRoute implements a modular architecture where each stage of request processing is isolated into small, testable functions. Understanding this flow is essential for developers extending the proxy, debugging provider-specific issues, or implementing custom middleware. This guide traces the exact path from HTTP request to upstream executor invocation using the `release/v3.8.50` branch.

## Overview of the Request Pipeline

A POST to `/v1/chat/completions` (or `/v1/responses`) traverses three architectural layers:

1. **Public Route Layer** – CORS, admission, and initial validation
2. **Handler Layer** – Core orchestration via `handleChatCore`
3. **Executor Layer** – Provider-specific upstream communication

Each layer delegates to the next through explicit function calls, with no implicit middleware magic.

## Layer 1: Next.js Public Route and Admission

The entry point is [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) (lines 71-99). This thin route handler performs protocol-level concerns before delegating to business logic:

- **CORS preflight** handling for cross-origin requests
- **Content-Type validation** ensuring proper JSON bodies
- **Heavy-weight admission** checks for capacity limits and request size
- **Prompt-injection guard** – first line of defense against jailbreaking attempts
- **Streaming detection** from the `stream` parameter to determine response handling strategy

Once validated, the route calls `handleChat` from `@/sse/handlers/chat`, which immediately forwards to `handleChatCore`.

```typescript
// From src/app/api/v1/chat/completions/route.ts (simplified)
import { handleChat } from "@/sse/handlers/chat";

export async function POST(request: Request) {
  // CORS, validation, admission checks...
  const response = await handleChat({ body, modelInfo, credentials, log });
  return new Response(response.body, { headers: response.headers });
}

```

## Layer 2: Core Handler Orchestration in chatCore.ts

[`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) contains `handleChatCore`, the central orchestrator that sequences 13+ discrete processing steps. Each sub-step resides in its own file for isolated testing.

### Step 2a: Request Setup and Routing Metadata

[`requestSetup.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/requestSetup.ts) extracts routing decisions from the incoming request:

- Target **provider** (openai, anthropic, gemini, etc.)
- **Model** identifier and **combo** routing rules
- **Extended context** flags for special handling modes

### Step 2b: Format Resolution

Two complementary modules determine transformation requirements:

- **[`requestFormat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/requestFormat.ts)** – Infers the **source format** from request shape (OpenAI-compatible, Anthropic Messages, Gemini, etc.)
- **[`targetFormat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/targetFormat.ts)** – Determines the **target format** the selected provider expects

This dual-resolution enables OmniRoute to translate between arbitrary provider schemas.

### Step 2c: Early Cache Short-Circuits

Before expensive processing, two cache layers are consulted:

- **Idempotency cache** ([`idempotency.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/idempotency.ts)) – Returns identical responses for replayed requests with the same idempotency key
- **Semantic cache** ([`semanticCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/semanticCache.ts)) – Vector-similarity matching for semantically equivalent prompts

Either can return a cached response and terminate the pipeline early.

### Step 2d: Guardrails and Plugin Hooks

Custom policy enforcement runs via:

- **`runPluginOnRequestHook`** – Transform or block requests based on configured plugins
- **`runPluginOnResponseHook`** – Scheduled for post-processing (invoked after upstream response)

### Step 2e: Memory and Skills Injection

[`memorySkillsInjection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/memorySkillsInjection.ts) augments the request body with:

- **Persistent memory** from prior conversations
- **Skill call definitions** for A2A (agent-to-agent) capabilities

### Step 2f: Proactive Prompt Compression

When prompts exceed ~70% of the model's token limit, [`compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/compression/strategySelector.ts) executes a tiered pipeline:

| Stage | Strategy | Intensity |
|-------|----------|-----------|
| 1 | Lite | Minimal semantic preservation |
| 2 | Standard | Balanced compression |
| 3 | Caveman | Aggressive simplification |
| 4 | Aggressive | Maximum token reduction |

The body is replaced with the compressed version if any stage succeeds.

### Step 2g-h: Translation and Upstream Body Assembly

- **`translateRequest`** (from [`translator/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/translator/index.ts)) converts the unified internal schema to provider-specific JSON
- **[`upstreamBody.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/upstreamBody.ts)** assembles the exact payload for the HTTP request to the LLM provider

### Step 2i: Header Construction

[`upstreamExecuteHeaders.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/upstreamExecuteHeaders.ts) builds per-request headers including:

- Authentication tokens (`Authorization`, `x-api-key`, etc.)
- Custom User-Agent strings
- Cache-control overrides
- Provider-specific metadata headers

### Step 2j-k: Executor Selection and Provider Call

```typescript
// From open-sse/executors/index.ts
import { getExecutor } from "@/sse/executors";

const executor = getExecutor(providerId);  // "openai" → DefaultExecutor
const upstreamResponse = await executor.execute({
  url: providerEndpoint,
  headers: upstreamHeaders,
  body: upstreamBody,
});

```

The `getExecutor` factory (in [`open-sse/executors/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/index.ts)) selects the appropriate class:

- **`DefaultExecutor`** – OpenAI-compatible providers (most common)
- **`CursorExecutor`** – Cursor-specific handling
- **`CodexExecutor`** – OpenAI Codex endpoints
- **`VertexExecutor`** – Google Cloud Vertex AI

All executors inherit from `BaseExecutor` ([`open-sse/executors/base.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/base.ts)), which implements:

- URL construction with path templates
- Request transformer application
- **Fetch with retry/back-off**
- **Timeout enforcement**
- **Circuit-breaker** logic for failing providers

### Step 2l: Response Handling Paths

**Streaming responses** (`stream: true`) flow through [`streamingPipeline.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/streamingPipeline.ts):

```typescript
// Simplified streaming pipeline assembly
const transformStream = createSSETransformStreamWithLogger(log);
const usageStream = streamingUsageStats(modelInfo);
const costStream = streamingCost(pricing);
// TransformStream chain: raw bytes → parsed SSE → usage accounting → cost tracking → client

```

- `createSSETransformStreamWithLogger` – Parses raw bytes to SSE events with logging
- `createPassthroughStreamWithLogger` – Optional debugging passthrough
- Heartbeat injection for connection keepalive
- [`streamingResponseHeaders.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/streamingResponseHeaders.ts) – Final HTTP header assembly

**Non-streaming responses** use:

- [`nonStreamingResponseParse.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/nonStreamingResponseParse.ts) – Full body consumption
- [`nonStreamingResponseHeaders.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/nonStreamingResponseHeaders.ts) – Header construction
- Synchronous usage and cost recording

### Step 2m: Post-Processing and Analytics

Before returning to the client:

- **Semantic cache storage** for future similar requests
- **Compressed usage receipt** attachment in response metadata
- **Quota-share** updates for multi-tenant accounting
- **Gamification event** emission for engagement tracking

## Layer 3: Responses API Wrapper

The `/v1/responses` endpoint (OpenAI's newer API shape) uses [`responsesHandler.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/responsesHandler.ts) as an adapter:

1. Convert incoming **Responses API format** to **Chat Completions format**
2. Call `handleChatCore` (steps 2a-2m above)
3. Transform the upstream SSE stream back via `createResponsesApiTransformStream`

This avoids code duplication while supporting both API specifications.

## Complete Request Flow Visualization

```

POST /v1/chat/completions
        │
        ▼
┌─────────────────┐
│  route.ts       │ CORS, admission, prompt-injection guard
│  (lines 71-99)  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  handleChat     │ Thin wrapper
│  → handleChatCore│
└────────┬────────┘
         │
    ┌────┴────┬────────┬────────┬────────┐
    ▼         ▼        ▼        ▼        ▼
requestSetup  format   caches   plugins  memory/
              resolution        guardrails skills
    └────┬────┴────────┴────────┴────────┘
         ▼
    compression/strategySelector.ts (if needed)
         │
         ▼
    translator/index.ts ──► upstreamBody.ts ──► upstreamExecuteHeaders.ts
         │
         ▼
    executors/index.ts → getExecutor(provider) → executor.execute()
                                                    │
                                                    ▼
                                              LLM Provider API
                                                    │
         ┌──────────────────────────────────────────┘
         ▼
    streamingPipeline.ts OR non-streaming handlers
         │
         ▼
    semantic cache store, quota update, analytics
         │
         ▼
    Response to client

```

## Code Examples

### Minimal cURL to Exercise Full Pipeline

```bash
curl -X POST https://omniroute.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMNI_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Explain request routing"}],
    "stream": true
  }'

```

### Direct Core Handler Invocation (Testing)

```typescript
import { handleChatCore } from "@/sse/handlers/chatCore";

const response = await handleChatCore({
  body: {
    model: "claude-3-opus-20240229",
    messages: [{ role: "user", content: "Hello" }],
    stream: false,
  },
  modelInfo: { provider: "anthropic", model: "claude-3-opus-20240229" },
  credentials: { apiKey: process.env.ANTHROPIC_API_KEY },
  log: console,
});
// response is a native Response object ready to return to client

```

### Executor Factory Usage

```typescript
import { getExecutor } from "@/sse/executors";

async function callProvider(body: unknown, provider: string, url: string) {
  const executor = getExecutor(provider);  // DefaultExecutor, VertexExecutor, etc.
  
  return executor.execute({
    url,
    headers: { Authorization: `Bearer ${getApiKey(provider)}` },
    body,
    timeoutMs: 60000,
  });
}

```

## Key Source Files Reference

| File | Responsibility | Branch Link |
|------|---------------|-------------|
| [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) | Public API surface, admission | [route.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/app/api/v1/chat/completions/route.ts) |
| [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) | Central orchestration logic | [chatCore.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/handlers/chatCore.ts) |
| [`open-sse/handlers/chatCore/requestSetup.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore/requestSetup.ts) | Routing metadata extraction | [requestSetup.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/handlers/chatCore/requestSetup.ts) |
| [`open-sse/handlers/chatCore/idempotency.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore/idempotency.ts) | Request deduplication | [idempotency.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/handlers/chatCore/idempotency.ts) |
| [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) | Prompt compression pipeline | [strategySelector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/strategySelector.ts) |
| [`open-sse/translator/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/index.ts) | Cross-provider schema translation | [translator/index.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/translator/index.ts) |
| [`open-sse/executors/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/index.ts) | Executor factory | [executors/index.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/executors/index.ts) |
| [`open-sse/executors/base.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/base.ts) | Base executor with retry/circuit logic | [base.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/executors/base.ts) |
| [`open-sse/handlers/chatCore/streamingPipeline.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore/streamingPipeline.ts) | SSE stream transformation | [streamingPipeline.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/handlers/chatCore/streamingPipeline.ts) |

## Summary

The **OmniRoute open-SSE request flow** implements a deliberately granular pipeline:

- **Admission and guards** run at the edge before expensive processing
- **Cache layers** can short-circuit 80%+ of identical requests
- **Compression and translation** adapt arbitrary prompts to provider constraints
- **Executor abstraction** unifies retry, timeout, and circuit-breaker semantics across dozens of LLM providers
- **Streaming and non-streaming paths** share core logic but apply appropriate response handling

This architecture enables **fine-grained testing** (each file has unit tests), **runtime observability** (every stage emits structured logs), and **safe extensibility** (new providers add an executor class; new policies inject at plugin hooks).

## Frequently Asked Questions

### How does OmniRoute handle provider-specific API differences?

OmniRoute uses **two-phase translation**: [`requestFormat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/requestFormat.ts) and [`targetFormat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/targetFormat.ts) identify source and target schemas, then [`translator/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/translator/index.ts) applies the transformation. Provider-specific executors (`open-sse/executors/`) handle URL patterns, authentication schemes, and response parsing that can't be generalized.

### What happens when a request exceeds token limits?

The [`compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/compression/strategySelector.ts) module proactively compresses prompts exceeding ~70% of the model's context window. It attempts four strategies in ascending intensity (lite → standard → caveman → aggressive), stopping at the first that brings the prompt under threshold. Each strategy is a pure function tested in isolation.

### Can the pipeline be extended with custom middleware?

Yes. The **plugin system** exposes `runPluginOnRequestHook` and `runPluginOnResponseHook` injection points in `open-sse/handlers/chatCore/`. Plugins receive the full request context and can modify, log, or reject requests. Custom executors can be registered in [`executors/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/executors/index.ts) for provider-specific behavior.