# Understanding the Request Pipeline Architecture in OmniRoute: From HTTP Request to Streamed Response

> Explore OmniRoute's request pipeline architecture. Discover how it transforms HTTP requests into provider-agnostic streaming responses through 11 stages for enhanced security and capacity management.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-08-23

---

**OmniRoute’s request pipeline architecture processes incoming chat completion requests through 11 distinct stages, transforming HTTP calls into provider-agnostic streaming responses while enforcing security, capacity limits, and usage tracking.**

OmniRoute is an open-source AI gateway that normalizes requests across multiple LLM providers. Its request pipeline architecture handles everything from CORS validation to token usage persistence, ensuring reliable delivery of chat completions through a modular, middleware-based design.

## The 11-Stage Request Processing Flow

OmniRoute’s pipeline handles a **POST** request to `/v1/chat/completions` through sequential stages, each isolated in specific modules for maintainability and observability.

### 1. HTTP Entry and CORS Validation

The pipeline begins at [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts), where the route handler validates Content-Type headers and handles CORS pre-flight requests. Non-JSON bodies receive an immediate **415** response, while valid requests proceed to admission control.

### 2. Admission and Capacity Throttling

Before processing, [`src/shared/middleware/chatBodyAdmission.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/middleware/chatBodyAdmission.ts) enforces capacity limits via the `admitChatRequest` function. This stage builds a **request-lease** that auto-releases when the request completes, protecting against DoS attacks through early-exit overload protection.

### 3. Prompt-Injection Guard

Security scanning occurs in [`src/middleware/promptInjectionGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/middleware/promptInjectionGuard.ts) through `createInjectionGuard`. Malicious payloads trigger an immediate **400** response before reaching downstream providers.

### 4. Model-Alias Resolution

User-friendly model names (e.g., `gpt-4o`) normalize to provider-specific identifiers in [`src/lib/modelAliasResolver.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/modelAliasResolver.ts). The `resolveModelAliasWithSeedFallbackOnBody` function maintains a seeded list of 30+ aliases for automatic fallback when unknown models are requested.

### 5. Streaming Decision

The pipeline determines response format in [`src/shared/utils/aiSdkCompat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/aiSdkCompat.ts). The `acceptHeaderForcesStream` utility checks for `stream:true` or SSE Accept headers, preparing the early-keep-alive wrapper from [`src/open-sse/utils/earlyStreamKeepalive.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/open-sse/utils/earlyStreamKeepalive.ts) for long-running connections.

### 6. Core Chat Handler

The `handleChat` function in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) orchestrates combo logic, provider credential selection, and fallback loops. This stage dispatches prepared requests to the SSE core while handling error boundaries.

### 7. SSE Core Orchestration

[`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) contains the `handleChatCore` function, which translates requests to provider-specific formats using translators in `open-sse/translator/*`. This provider-agnostic layer enables seamless switching between OpenAI, Claude, Gemini, and other backends.

### 8. Provider Executor

Provider-specific executors in `open-sse/executors/*` (e.g., `DefaultExecutor`, `AntigravityExecutor`) extend the `BaseExecutor` pattern defined in [`open-sse/executors/base.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/base.ts). These handle HTTP dispatch, retries, token refresh, and circuit-breaker logic while respecting per-provider rate limits.

### 9. Stream Transformation

Response normalization occurs in [`open-sse/utils/stream.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/stream.ts) and [`streamHandler.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/streamHandler.ts), converting upstream SSE/JSON into OpenAI-compatible formats. The [`open-sse/handlers/responseSanitizer.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/responseSanitizer.ts) removes unsafe fields and injects usage metadata.

### 10. Usage Extraction and Persistence

Token counting and cost tracking happen in [`open-sse/utils/usageTracking.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/usageTracking.ts), persisting records through [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts). This captures provider, model, prompt tokens, and completion timestamps for audit logs and dashboards.

### 11. Response Wrapping

Final HTTP response construction adds correlation IDs and compression headers via [`src/shared/utils/compressionHeaderEcho.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/compressionHeaderEcho.ts). For streaming requests, the keep-alive wrapper ensures connection stability throughout the lifecycle.

## Key Implementation Details

### Entry Point and Request Admission

The route handler in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) wires the pipeline together:

```typescript
import { handleChat } from "@/sse/handlers/chat";
import { generateRequestId } from "@/shared/utils/requestId";
import { admitChatRequest } from "@/shared/middleware/chatBodyAdmission";
import { acceptHeaderForcesStream } from "@/shared/utils/aiSdkCompat";
import { withEarlyStreamKeepalive } from "@/open-sse/utils/earlyStreamKeepalive";

export async function POST(request: Request) {
  await ensureInitialized();
  const admission = await admitChatRequest(request, { 
    sessionId: resolveSessionId(request) 
  });
  const parsedBody = await request.json();
  
  const wantsStreaming = parsedBody?.stream === true || 
    acceptHeaderForcesStream(request.headers.get("accept"));
  const reqId = generateRequestId();

  if (wantsStreaming) {
    const handler = handleChat(request, null, parsedBody, reqId);
    return withEarlyStreamKeepalive(handler);
  }

  return handleChat(request, null, parsedBody);
}

```

### Provider Translation and Execution

The SSE core handles provider abstraction through translation layers:

```typescript
// open-sse/handlers/chatCore.ts
export async function handleChatCore(
  body: ChatRequest,
  modelInfo: ResolvedModel,
  credentials: ProviderCredentials,
  requestId: string
) {
  const translated = await translateRequest(body, modelInfo);
  const execResult = await getExecutor(modelInfo.provider)
    .execute(translated, credentials);
  const stream = await translateResponse(execResult, requestId);
  return stream;
}

```

### Usage Tracking Implementation

After provider response, usage extraction enables cost management:

```typescript
// open-sse/utils/usageTracking.ts
export async function extractUsage(
  stream: ReadableStream, 
  meta: { provider: string; model: string }
) {
  const usage = await parseUsageFromStream(stream);
  await usageDb.record({
    provider: meta.provider,
    model: meta.model,
    promptTokens: usage.promptTokens,
    completionTokens: usage.completionTokens,
    timestamp: Date.now(),
  });
}

```

## Architecture Benefits

**OmniRoute’s request pipeline architecture** provides several operational advantages:

- **Provider Agnosticism**: The translation layer in `open-sse/translator/*` decouples client requests from provider-specific formats, enabling hot-swapping between LLM backends without client changes.
- **Resilience Patterns**: Base executors implement circuit-breakers, exponential backoff, and automatic token refresh, isolating failures to specific providers while maintaining service availability.
- **Security-First Design**: Prompt-injection scanning and response sanitization occur at pipeline boundaries, ensuring malicious content never reaches upstream providers or downstream clients.
- **Observability**: Correlation IDs flow through all 11 stages, with usage persistence enabling per-request cost attribution and capacity planning.

## Summary

- OmniRoute processes chat completions through 11 distinct pipeline stages, from HTTP entry in [`route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/route.ts) to usage persistence in [`usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/usageDb.ts).
- **Capacity protection** via `admitChatRequest` prevents overload, while **prompt-injection guards** block malicious payloads before provider dispatch.
- **Model-alias resolution** normalizes user-friendly names to provider IDs, supporting 30+ pre-seeded aliases with automatic fallback.
- Provider executors extend a **BaseExecutor** pattern that standardizes retries, authentication refresh, and rate-limit compliance across OpenAI, Claude, Gemini, and custom backends.
- Stream transformations guarantee **OpenAI SDK compatibility** regardless of upstream provider format, with keep-alive frames maintaining long-running connections.
- Usage tracking captures token counts and metadata for every request, enabling cost attribution and audit trails through [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts).

## Frequently Asked Questions

### How does OmniRoute handle high-traffic load and prevent server overload?

OmniRoute implements **admission control** in [`src/shared/middleware/chatBodyAdmission.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/middleware/chatBodyAdmission.ts) through the `admitChatRequest` function. This middleware creates a request-lease for each incoming connection, enforcing per-session limits and global capacity thresholds. When capacity is exceeded, the pipeline returns early with an overload response, protecting downstream providers and maintaining service stability for admitted requests.

### What security measures exist in the request pipeline to prevent prompt injection attacks?

The pipeline includes a dedicated **prompt-injection guard** stage implemented in [`src/middleware/promptInjectionGuard.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/middleware/promptInjectionGuard.ts). Before any downstream processing occurs, the `createInjectionGuard` function scans the JSON payload for malicious patterns. If injection attempts are detected, the pipeline terminates immediately with a **400** response, ensuring harmful content never reaches LLM providers or training data.

### Can OmniRoute translate between different LLM provider formats automatically?

Yes. The **SSE Core Orchestration** stage in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) uses provider-specific translators located in `open-sse/translator/*` to convert requests and responses bidirectionally. This means clients can use standard OpenAI SDK formats while the gateway automatically translates to Claude, Gemini, or other provider schemas. The `translateRequest` and `translateResponse` functions handle format normalization, enabling seamless provider switching without client code changes.

### Where does usage tracking occur in the pipeline, and what data is captured?

Usage extraction happens in stage 10 through [`open-sse/utils/usageTracking.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/usageTracking.ts) after the provider response is received but before final client delivery. The `extractUsage` function captures provider name, model identifier, prompt tokens, completion tokens, and timestamps. This data persists to [`src/lib/usageDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usageDb.ts), enabling real-time cost tracking, usage dashboards, and comprehensive audit logs for every chat completion request.