# OmniRoute Request Pipeline Architecture: 11-Stage Flow from HTTP to Streamed Response

> Explore OmniRoute's 11 stage request pipeline architecture. Understand how HTTP calls transform into streamed or JSON responses for chat completions.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-08-25

---

**OmniRoute's request pipeline transforms incoming `/v1/chat/completions` calls into streaming or JSON responses through 11 sequential stages spanning authentication, policy enforcement, model resolution, provider translation, and usage tracking.**

Built by **diegosouzapw/OmniRoute**, this pipeline delivers a **provider-agnostic LLM gateway** that normalizes requests across OpenAI, Claude, Gemini, and other providers while maintaining strict OpenAI SDK compatibility. The architecture emphasizes **resilience, observability, and security** at every layer.

---

## Pipeline Overview: From HTTP Entry to Final Response

The complete flow executes in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) and delegates through specialized modules. Each stage is designed to fail fast, preserve context, and release resources properly.

---

## Stage 1: HTTP Entry and CORS Handling

The `POST` and `OPTIONS` handlers in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) validate **Content-Type** headers and enforce CORS pre-flight checks. Non-JSON bodies immediately receive a **415 Unsupported Media Type** response.

```typescript
// Entry point from route.ts
export async function POST(request: Request) {
  await ensureInitialized(); // translators initialization
  const parsedBody = await request.json();
  // ... downstream processing
}

```

---

## Stage 2: Admission and Capacity Throttling

**`admitChatRequest`** and **`admitChatStructure`** in [`src/shared/middleware/chatBodyAdmission.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/middleware/chatBodyAdmission.ts) guarantee system capacity through **request-leases**. These leases are acquired before processing and released on completion, protecting against DoS attacks and overload scenarios.

- Early-exit on capacity exhaustion
- Per-session request limits
- Automatic lease cleanup via `finally` blocks

---

## Stage 3: Prompt-Injection Guard

Before any downstream processing, **`createInjectionGuard`** in [`src/middleware/promptInjectionGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/middleware/promptInjectionGuard.ts) scans the JSON payload for malicious patterns. Blocked requests return **400 Bad Request** with security context.

---

## Stage 4: Model-Alias Resolution

**`resolveModelAliasWithSeedFallbackOnBody`** in [`src/lib/modelAliasResolver.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/modelAliasResolver.ts) converts user-friendly names like `gpt-4o` into provider-specific identifiers. The system maintains a **seeded fallback list of 30+ aliases** for unknown model names, ensuring graceful degradation.

| Input Alias | Resolved Provider String |
|-------------|--------------------------|
| `gpt-4o` | `openai/gpt-4o-2024-05-13` |
| `claude-3-opus` | `anthropic/claude-3-opus-20240229` |
| `gemini-pro` | `google/gemini-1.0-pro` |

---

## Stage 5: Streaming Decision

The pipeline detects streaming intent through:

- **`parsedBody.stream === true`** in request body
- **`acceptHeaderForcesStream`** in [`src/shared/utils/aiSdkCompat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/aiSdkCompat.ts) checking the `Accept` header

When streaming is requested, **`withEarlyStreamKeepalive`** from [`src/open-sse/utils/earlyStreamKeepalive.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/open-sse/utils/earlyStreamKeepalive.ts) wraps the handler to inject **keep-alive frames** preventing timeouts on long-running generations.

```typescript
const wantsStreaming = parsedBody?.stream === true 
  || acceptHeaderForcesStream(request.headers.get("accept"));

if (wantsStreaming) {
  const handler = handleChat(request, null, parsedBody, reqId);
  return withEarlyStreamKeepalive(handler, { maxDurationMs: 30000 });
}

```

---

## Stage 6: Core Chat Handler Orchestration

**`handleChat`** in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) performs the heavy lifting:

- Parses and validates request body
- Resolves **combo** configurations (multi-provider fallbacks)
- Selects appropriate provider credentials
- Dispatches to the SSE core with error handling loops

This stage handles **fallback logic** when primary providers fail, cycling through alternatives automatically.

---

## Stage 7: SSE Core and Provider Translation

**`handleChatCore`** in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) operates as the **provider-agnostic translation layer**:

```typescript
export async function handleChatCore(
  body: ChatRequest,
  modelInfo: ResolvedModel,
  credentials: ProviderCredentials,
  requestId: string
) {
  const translated = await translateRequest(body, modelInfo);
  const execResult = await getExecutor(modelInfo.provider).execute(translated, credentials);
  const stream = await translateResponse(execResult, requestId);
  return stream;
}

```

The **`open-sse/translator/`** directory contains request/response converters for each supported provider, enabling seamless format translation between OpenAI, Claude, Gemini, and others.

---

## Stage 8: Provider Execution with Resilience

Each provider implements an executor in `open-sse/executors/`:

| Executor | Provider | Specialization |
|----------|----------|--------------|
| `DefaultExecutor` | Generic OpenAI-compatible | Standard retry and back-off |
| `AntigravityExecutor` | Antigravity AI | Custom token refresh handling |
| `BaseExecutor` | All (abstract) | Circuit-breaker, rate-limit respect, cooldown |

The **`BaseExecutor`** pattern centralizes:
- **Exponential backoff** with jitter
- **OAuth token refresh** before expiry
- **Circuit-breaker** state management
- **Per-provider rate limit** tracking

---

## Stage 9: Stream Transformation and Sanitization

**[`open-sse/utils/stream.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/stream.ts)** and **[`streamHandler.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/streamHandler.ts)** normalize upstream SSE streams into **strict OpenAI-compatible format**. The **`responseSanitizer`** removes unsafe fields and injects proper SSE framing.

Key transformations:
- Provider-specific event formats → OpenAI SSE
- Usage metadata injection
- Response field filtering per security policies

---

## Stage 10: Usage Extraction and Persistence

Post-execution, **`extractUsage`** in [`open-sse/utils/usageTracking.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/usageTracking.ts) parses token counts from the stream. **[`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts)** persists records for **cost tracking dashboards and audit logs**:

```typescript
export async function extractUsage(stream: ReadableStream, meta: { provider: string; model: string }) {
  const usage = await parseUsageFromStream(stream);
  await usageDb.record({
    provider: meta.provider,
    model: meta.model,
    promptTokens: usage.promptTokens,
    completionTokens: usage.completionTokens,
    timestamp: Date.now(),
  });
}

```

---

## Stage 11: Response Wrapping and Delivery

Final stage responsibilities:
- **Correlation ID** attachment (`generateRequestId`)
- **Compression header echo** via [`src/shared/utils/compressionHeaderEcho.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/compressionHeaderEcho.ts)
- **Keep-alive wrapper termination** for clean stream closure
- Proper `Content-Type` and caching headers

---

## Key Files and Their Responsibilities

| File Path | Pipeline Role |
|-----------|---------------|
| [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) | HTTP entry, CORS, admission orchestration |
| [`src/shared/middleware/chatBodyAdmission.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/middleware/chatBodyAdmission.ts) | Capacity throttling and lease management |
| [`src/middleware/promptInjectionGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/middleware/promptInjectionGuard.ts) | Security filtering |
| [`src/lib/modelAliasResolver.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/modelAliasResolver.ts) | Model name normalization |
| [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) | Combo handling and fallback orchestration |
| [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) | Translation and executor dispatch |
| [`open-sse/translator/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/index.ts) | Provider converter registry |
| [`open-sse/executors/base.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/base.ts) | Shared resilience logic |
| `open-sse/executors/*.ts` | Provider-specific execution |
| [`open-sse/utils/earlyStreamKeepalive.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/earlyStreamKeepalive.ts) | Streaming timeout prevention |
| [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts) | Cost and audit persistence |
| [`docs/architecture/ARCHITECTURE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/architecture/ARCHITECTURE.md) | Architecture documentation and diagrams |

---

## Summary

- **11 sequential stages** process every `/v1/chat/completions` request in OmniRoute
- **Fail-fast security** via prompt-injection guards and capacity throttling
- **Provider abstraction** through translation layers and executor patterns
- **Resilience built-in**: circuit-breakers, retries, fallbacks, and keep-alive mechanisms
- **Full observability**: correlation IDs, usage tracking, and audit logging
- **OpenAI SDK compatibility** maintained regardless of upstream provider

The pipeline's modular design allows individual stages to evolve independently while preserving the **contract between HTTP entry and streamed response**.

---

## Frequently Asked Questions

### What triggers streaming mode in OmniRoute?

Streaming activates when the request body contains `"stream": true` **or** when the `Accept` header indicates SSE preference (detected by `acceptHeaderForcesStream` in [`src/shared/utils/aiSdkCompat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/aiSdkCompat.ts)). The `withEarlyStreamKeepalive` wrapper then ensures the connection remains alive during long generations.

### How does OmniRoute handle provider failures?

The **`handleChat`** function in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) implements **combo logic** with automatic fallback loops. If the primary provider's executor throws, the pipeline cycles through configured alternatives. Each executor inherits resilience patterns from `BaseExecutor` including circuit-breakers and exponential backoff.

### Where are model aliases defined and resolved?

Aliases resolve in [`src/lib/modelAliasResolver.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/modelAliasResolver.ts) via `resolveModelAliasWithSeedFallbackOnBody`. The system maps friendly names like `gpt-4o` to provider-specific identifiers, falling back to a **seeded list of 30+ aliases** when encountering unknown values. This enables provider-agnostic client requests.

### How is token usage tracked for billing?

The **`extractUsage`** utility in [`open-sse/utils/usageTracking.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/usageTracking.ts) parses token counts from provider responses. **[`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts)** persists records with provider, model, prompt tokens, completion tokens, and timestamp. This data powers cost-tracking dashboards and satisfies audit requirements.