# How OmniRoute Structures Its OpenAI-Compatible API Across Endpoints

> Discover how OmniRoute structures its OpenAI-compatible API using Next.js App Router. Explore endpoint delegation, modular handlers, and robust validation for efficient AI integration.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-08-06

---

**OmniRoute exposes a unified OpenAI-compatible REST API under `/api/v1/` using Next.js App Router, with each endpoint delegating to modular handlers in `open-sse/handlers/` after CORS guards, Zod validation, and policy checks.**

This architecture lets developers call 300+ LLM providers through a single interface that mirrors OpenAI's official surface. The diegosouzapw/OmniRoute repository implements this through five pipeline stages: method guarding, request validation, authentication, handler delegation, and provider-specific execution.

## Core Pipeline Architecture

Every OpenAI-compatible route in OmniRoute follows the same execution flow. Understanding this pipeline explains how the system maintains consistency across diverse endpoints.

### Stage 1: CORS and Method Guarding

Route files export `GET` or `POST` handlers that Next.js wires automatically. Each handler begins with CORS headers and HTTP method validation before processing continues.

### Stage 2: Zod Schema Validation

Request bodies and query parameters undergo strict validation using Zod schemas. This catches malformed payloads early and returns standardized error responses matching OpenAI's format.

### Stage 3: Authentication and Policy Enforcement

The system extracts API keys via `extractApiKey`, then applies layered policies:

- **Rate limiting** per key and per provider
- **Prompt injection detection** guards
- **Circuit-breaker health checks** against known-failing providers

### Stage 4: Handler Delegation

Validated requests forward to core handlers in `open-sse/handlers/`:

- `chatCore` for chat completions
- `completionCore` for legacy text completions
- `embeddingsCore` for vector generation
- `imagesCore` for DALL-E-style generation
- `audioCore` for Whisper transcription/translation
- `modelsCore` for catalogue queries

### Stage 5: Translation and Execution

Handlers translate OpenAI payloads to provider-specific formats using translators in `open-sse/translator/`, select executors from `open-sse/executors/`, and stream responses back via Server-Sent Events (SSE) when `stream: true`.

## Endpoint Reference: File Paths and Responsibilities

| OpenAI Concept | Route File | Core Handler | Purpose |
|:--|:--|:--|:--|
| **Chat Completions** | [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) | [`open-sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chat.ts) | Main streaming chat endpoint |
| **Text Completions** (legacy) | [`src/app/api/v1/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/completions/route.ts) | [`open-sse/handlers/completions.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/completions.ts) | Non-chat completion API |
| **Embeddings** | [`src/app/api/v1/embeddings/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/embeddings/route.ts) | [`open-sse/handlers/embeddings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/embeddings.ts) | Vector representation generation |
| **Image Generations** | [`src/app/api/v1/images/generations/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/images/generations/route.ts) | [`open-sse/handlers/images.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/images.ts) | DALL-E-compatible image creation |
| **Audio Transcriptions** | [`src/app/api/v1/audio/transcriptions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/audio/transcriptions/route.ts) | [`open-sse/handlers/audio.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/audio.ts) | Speech-to-text (Whisper) |
| **Audio Translations** | [`src/app/api/v1/audio/translations/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/audio/translations/route.ts) | [`open-sse/handlers/audio.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/audio.ts) | Cross-language transcription |
| **Audio Speech** | [`src/app/api/v1/audio/speech/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/audio/speech/route.ts) | [`open-sse/handlers/audio.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/audio.ts) | Text-to-speech generation |
| **Models List** | [`src/app/api/v1/models/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/models/route.ts) | [`open-sse/handlers/models.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/models.ts) | Available model catalogue |
| **Model Retrieval** | `src/app/api/v1/models/[...model]/route.ts` | [`open-sse/handlers/models.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/models.ts) | Single model details |

All these routes use [`open-sse/executors/openaiCompatible.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/openaiCompatible.ts) for execution unless provider-scoped alternatives apply.

## Provider-Scoped and Specialized Endpoints

### Forcing Provider Selection

The `/v1/providers/{provider}/` namespace bypasses automatic provider selection. Instead of letting OmniRoute choose based on model prefix, these endpoints force routing to a specific backend.

```bash
curl -X POST http://localhost:20128/v1/providers/anthropic/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-key" \
  -d '{
    "model": "anthropic/claude-3-5-sonnet",
    "messages": [{"role": "user", "content": "Write a haiku about spring."}]
  }'

```

The route at `src/app/api/v1/providers/[provider]/chat/completions/route.ts` pulls provider configuration from [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts), which defines base URLs, authentication schemes, and format flags (OpenAI, Claude, or Gemini).

### VS Code Token Alias

Extensions embedding OmniRoute can pass raw API keys in the URL path:

```bash
curl -X POST http://localhost:20128/api/v1/vscode/sk-abcdef12345/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

```

The handler at `src/app/api/v1/vscode/[token]/chat/completions/route.ts` extracts the token from the path segment, validates it as a raw key, then proceeds with standard chat handling.

### Relay Mode Endpoints

For debugging or custom proxy scenarios, [`src/app/api/v1/relay/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/chat/completions/route.ts) and [`src/app/api/v1/relay/bifrost/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/relay/bifrost/route.ts) provide bypass routes. These skip policy checks already performed upstream, supporting either TypeScript-native relay or the external Bifrost side-car proxy.

## Combo Routing and Auto-Selection

The `/v1/combos/` and `/v1/auto-combo/` endpoints implement intelligent request distribution:

```bash
curl -X POST http://localhost:20128/v1/auto-combo/default/candidates \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Summarize the plot of Inception."}]
  }'
`

The route `src/app/api/v1/auto-combo/[channel]/candidates/route.ts` delegates to `open-sse/services/combo.ts`, which applies strategies (auto, weighted, fusion) before invoking standard chat handlers with selected model targets.

## Streaming vs. JSON Response Handling

Every endpoint respects the `stream` parameter (default: `true`) and `Accept` header. When streaming is requested, OmniRoute:

1. Applies early keep-alive SSE guards to prevent connection timeouts
2. Uses chunked transfer encoding with `data:` prefixed SSE frames
3. Closes with `[DONE]` marker matching OpenAI's format

For `stream: false` or `Accept: application/json`, the executor buffers the complete response and returns standard JSON.

## Resilience Mechanisms

OmniRoute's OpenAI-compatible API incorporates failure handling at multiple layers:

- **Circuit breaker**: `src/shared/utils/circuitBreaker.ts` tracks provider health and excludes failing backends from selection
- **Account fallback**: `src/sse/services/accountFallback.ts` rotates credentials when rate limits or quota errors occur
- **Provider cooldown**: Automatic backoff for transient errors before re-enabling providers

These mechanisms operate outside route files, keeping endpoint code clean while ensuring high availability.

## Complete Code Examples

### Standard Chat Completion

```bash
curl -X POST http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-key" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Explain quantum tunneling in one sentence."}
    ],
    "temperature": 0.7
  }'

```

### Embedding Generation

```bash
curl -X POST http://localhost:20128/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-key" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": "The quick brown fox jumps over the lazy dog."
  }'

```

### Image Generation

```bash
curl -X POST http://localhost:20128/v1/images/generations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-key" \
  -d '{
    "prompt": "A futuristic city at sunset, photorealistic",
    "n": 2,
    "size": "1024x1024"
  }'

```

### Audio Transcription

```bash
curl -X POST http://localhost:20128/v1/audio/transcriptions \
  -H "Authorization: Bearer sk-your-key" \
  -F file=@speech.wav \
  -F model=whisper-1

```

## Key Files for API Extension

| File | Responsibility |
|:--|:--|
| [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts) | Central registry of 300+ providers with base URLs and format flags |
| [`open-sse/translator/openai-to-claude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/openai-to-claude.ts) | Example translator converting OpenAI schemas to Claude format |
| [`open-sse/translator/openai-to-gemini.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/openai-to-gemini.ts) | Google Gemini payload conversion |
| [`open-sse/executors/openaiCompatible.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/openaiCompatible.ts) | Generic executor for OpenAI-format providers |
| [`open-sse/executors/anthropic.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/executors/anthropic.ts) | Specialized executor for Anthropic's native API |
| [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) | Provider health tracking and automatic exclusion |

## Summary

- **Unified surface**: All endpoints live under `/api/v1/` with OpenAI-compatible request/response formats
- **Five-stage pipeline**: CORS guards → Zod validation → auth/policy → handler delegation → translation/execution
- **Modular handlers**: Core logic in `open-sse/handlers/` keeps route files thin and consistent
- **Provider abstraction**: Translators and executors in `open-sse/` convert between OpenAI schema and 300+ backend formats
- **Forced routing**: Provider-scoped endpoints (`/providers/{provider}/`) bypass automatic selection
- **Specialized paths**: VS Code aliases, relay modes, and combo routing extend baseline functionality
- **Resilience**: Circuit breakers and fallback logic operate transparently across all endpoints

## Frequently Asked Questions

### How does OmniRoute handle non-OpenAI providers like Anthropic or Gemini?

OmniRoute uses a **translator layer** in `open-sse/translator/` to convert incoming OpenAI-format requests to provider-native schemas. For Anthropic, [`openai-to-claude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/openai-to-claude.ts) transforms messages, parameters, and response formats. The executor layer then handles provider-specific authentication and streaming protocols, returning responses that match OpenAI's structure to the client.

### Can I disable automatic provider selection and force a specific backend?

Yes. Use the provider-scoped endpoint pattern: `/v1/providers/{provider}/chat/completions`. The route at `src/app/api/v1/providers/[provider]/chat/completions/route.ts` extracts the provider slug from the path, validates it against [`providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/providerRegistry.ts), and forces routing to that backend regardless of model prefix or load-balancing rules.

### What happens when a provider fails during request processing?

OmniRoute implements **circuit-breaker logic** via [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) that tracks error rates per provider. After threshold failures, a provider enters a cooldown period and is excluded from selection. For streaming requests, the system attempts **account fallback** through [`src/sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/accountFallback.ts) to rotate credentials before failing the request to the client.

### How do I add a new OpenAI-compatible endpoint to OmniRoute?

Create a new folder structure under `src/app/api/v1/` matching the desired path, add a [`route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/route.ts) that exports `GET`/`POST` handlers, and follow the five-stage pipeline. Import validation schemas from existing endpoints, delegate to or create a handler in `open-sse/handlers/`, and register any new provider requirements in [`open-sse/config/providerRegistry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/providerRegistry.ts).