How OmniRoute Handles Different Route Types: Architecture and Implementation

OmniRoute uses a unified middleware pipeline in Next.js App Router to handle diverse API route types—streaming chat, embeddings, images, and provider-specific endpoints—while enforcing consistent security, validation, and request transformation across all paths.

OmniRoute is an open-source AI routing layer built on Next.js App Router that standardizes how different API route types are processed. Whether handling real-time chat streaming or batch embedding requests, the system applies a common security and validation framework through specialized route.ts files located under src/app/api/v1/. This architecture ensures that every route type inherits the same protections while allowing specialized behavior for streaming, provider-specific protocols, and custom routing strategies.

Core Pattern: Unified Security and Validation Layers

Every route module in OmniRoute begins with a standardized boilerplate that enforces critical security and performance checks before request-specific logic executes. This pattern appears in files like src/app/api/v1/chat/completions/route.ts and src/app/api/v1/embeddings/route.ts.

CORS and Pre-Flight Handling

All routes export an OPTIONS handler that returns appropriate Access-Control-Allow-* headers. This enables cross-origin requests to any endpoint in the API surface, from chat completions to model catalog queries.

Content Validation and Payload Protection

The system implements strict input validation to prevent malformed requests and denial-of-service attacks:

• Content-Type Guard: Rejects non-JSON bodies with HTTP 415 (application/json required) as seen in lines 47-62 of src/app/api/v1/chat/completions/route.ts
• Heap-Pressure Admission: Early-rejects request bodies exceeding 256 KB to avoid OOM crashes (lines 64-71 in the same file)
• Prompt-Injection Guard: Runs request bodies through createInjectionGuard() before downstream processing (lines 84-107)

API Key Enforcement and Policy Guards

After validation, enforceApiKeyPolicy extracts and validates API keys, applying model limits and budget constraints. This enforcement appears consistently across routes including src/app/api/v1/embeddings/route.ts, ensuring uniform access control regardless of endpoint type.

Streaming vs. Non-Streaming Route Handling

OmniRoute distinguishes between streaming and non-streaming paths at the route level. The /v1/chat/completions endpoint is the only route that supports Server-Sent Events (SSE), while all other endpoints (embeddings, images, audio) return simple JSON responses.

In src/app/api/v1/chat/completions/route.ts, the streaming decision logic checks for explicit stream flags or forced accept headers:

const wantsStreaming = (parsedBodyIsRecord && parsedBody.stream === true) || acceptForcesStream;
if (wantsStreaming) {
  const streamedResponse = await withEarlyStreamKeepalive(
    handleChat(request, null, parsedBody, reqId),
    { signal: request.signal, thresholdMs: resolveKeepaliveThreshold(parsedBody?.model), extraHeaders: { "X-Correlation-Id": reqId } }
  );
  return withCompressionHeaderEcho(streamedResponse, compressionRequestHeader);
}

If streaming is not requested, the same handleChat function executes in "plain JSON" mode. This wrapper is reused by provider-specific routes like src/app/api/v1/providers/[provider]/chat/completions/route.ts, which inject the provider identifier while maintaining the same streaming logic.

Non-streaming routes such as embeddings skip the withEarlyStreamKeepalive wrapper and directly return JSON responses through createEmbeddingResponse.

Provider-Specific Routing Architecture

For multi-provider support, OmniRoute creates dedicated dynamic routes under src/app/api/v1/providers/[provider]/. These files act as thin wrappers that import shared utilities from the core implementation.

For example, src/app/api/v1/providers/[provider]/chat/completions/route.ts maps requests to /v1/providers/:provider/chat/completions while selecting the appropriate executor. The file imports handleChat from open-sse/handlers/chatCore.ts and simply injects the provider identifier into the request context, keeping the core logic provider-agnostic.

Similarly, src/app/api/v1/providers/[provider]/embeddings/route.ts forwards embedding requests to the embedding service while applying the same validation pipeline as the generic embedding endpoint.

Special-Purpose Route Types

Beyond standard chat and embedding endpoints, OmniRoute implements several specialized route categories:

Responses API

The /v1/responses endpoint transforms traditional chat-completion streams into the newer "Responses" format. Located in src/app/api/v1/responses/route.ts, this route uses responsesTransformer to convert SSE chunks into structured response objects while maintaining the same security middleware stack.

Relay Routes

Internal services use /v1/relay/chat/completions to bypass normal quota checks. The implementation in src/app/api/v1/relay/chat/completions/route.ts applies a per-IP rate limit (RELAY_IP_PER_MINUTE) instead of standard API key policies, allowing internal traffic to flow without consuming user budgets.

Combo Routing

The /v1/combos endpoint enables users to define custom routing strategies including priority-based, weighted, and round-robin selection. The handler in src/app/api/v1/combos/route.ts parses user-defined combo configurations and distributes requests across multiple providers according to the specified strategy.

Model Catalog and Utility Endpoints

The /v1/models endpoint lists available models with optional capability filtering (embedding, image, audio generation). MCP and A2A endpoints under open-sse/mcp-server/ and src/lib/a2a/ expose tool-based RPC services using the same middleware stack as the REST API, ensuring consistent authentication and validation.

The Request Pipeline Flow

Every route type in OmniRoute follows an identical processing sequence:

1. CORS and OPTIONS handling for pre-flight requests
2. Content-type and size validation (JSON only, 256 KB limit)
3. Prompt-injection guard via createInjectionGuard()
4. API-key extraction and policy enforcement via enforceApiKeyPolicy
5. Streaming decision (conditional, only for chat routes)
6. Core handler execution (handleChat, handleEmbedding, etc.) from open-sse/handlers/
7. Response translation through open-sse/translator/* to normalize provider-specific formats to OpenAI-compatible JSON
8. Compression-header echo and final Response object return

This pipeline is visualized in docs/diagrams/request-pipeline.mmd, anchored on the chat/completions route implementation.

Summary

• Unified Security: Every route type in OmniRoute inherits CORS handling, content validation, prompt-injection guards, and API key enforcement through a shared middleware pattern.
• Streaming Specialization: Only chat completion routes support SSE streaming via withEarlyStreamKeepalive, while other endpoints (embeddings, images, audio) use standard JSON responses.
• Provider Abstraction: Dynamic routes under providers/[provider]/ inject provider identifiers into provider-agnostic core handlers like handleChat and createEmbeddingResponse.
• Specialized Endpoints: The architecture supports relay routes for internal traffic, combo routing for custom strategies, and the Responses API for modern chat formats without duplicating security logic.
• Consistent Pipeline: All routes follow the same eight-step processing flow from validation to response translation, ensuring predictable behavior across the entire API surface.

Frequently Asked Questions

How does OmniRoute decide whether to stream a response?

OmniRoute checks the stream parameter in the request body or the Accept header to determine if the client wants Server-Sent Events. In src/app/api/v1/chat/completions/route.ts, if wantsStreaming evaluates to true, the handler wraps the execution in withEarlyStreamKeepalive to maintain the connection; otherwise, it returns a complete JSON payload. Only chat completion endpoints support this streaming path.

What is the difference between the generic chat endpoint and provider-specific routes?

The generic /v1/chat/completions endpoint routes requests based on model selection logic, while /v1/providers/[provider]/chat/completions explicitly targets a specific provider. The provider-specific route file is a thin wrapper that injects the provider identifier into the request context before calling the same handleChat function from open-sse/handlers/chatCore.ts, ensuring consistent processing while allowing direct provider access.

How does OmniRoute protect against prompt injection attacks?

Every route runs request bodies through createInjectionGuard() before processing, as implemented in lines 84-107 of src/app/api/v1/chat/completions/route.ts. This guard analyzes the content for malicious patterns or injection attempts before the data reaches the LLM provider, providing a consistent security layer across chat, embedding, and other endpoint types.

Can I use OmniRoute for non-OpenAI providers like Anthropic or Google?

Yes. The provider-specific routing architecture supports Anthropic, Google Gemini, and other LLM providers through dedicated path segments. When you call /v1/providers/anthropic/chat/completions, the route automatically selects Anthropic's executor and uses the translation layer in open-sse/translator/* to convert between provider-specific formats and OpenAI-compatible JSON, allowing seamless multi-provider integration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}