What Is the OpenAI-Compatible Endpoint for OmniRoute?

The OpenAI-compatible endpoint for OmniRoute is /v1/chat/completions, functioning as a drop-in replacement for OpenAI's Chat Completion API that processes requests through a seven-stage pipeline including CORS handling, validation, security guards, and provider dispatch.

OmniRoute is an open-source AI model router that exposes a fully OpenAI-compatible HTTP API, enabling existing applications to switch providers without modifying client code. The primary entry point for chat-based interactions is the /v1/chat/completions endpoint, implemented in the Next.js API routes layer. This endpoint accepts standard OpenAI request payloads and returns responses in the identical JSON schema, including full support for Server-Sent Events (SSE) streaming.

Endpoint Overview and Path

The canonical path for chat completions is /v1/chat/completions, exposed as an HTTP POST endpoint. When running locally, clients target http://localhost:20128/v1/chat/completions (or the equivalent host/port configuration).

In src/app/api/v1/chat/completions/route.ts, the route implements the complete OpenAI contract: it accepts the standard payload shape containing model and messages parameters, returns identical JSON response schemas, and respects the stream flag for toggling between single-shot and streaming responses. The implementation ensures that any OpenAI-compatible client—from curl to the official OpenAI SDK—can communicate with OmniRoute without configuration changes.

Request Processing Pipeline

Every request to /v1/chat/completions flows through a strict seven-stage pipeline defined in src/app/api/v1/chat/completions/route.ts:

1. CORS Handling: handleCorsOptions processes OPTIONS pre-flight requests and sets appropriate cross-origin headers using utilities from src/shared/utils/cors.ts.

2. Request Validation: A permissive Zod schema named chatCompletionsRouteShapeSchema validates that the body is an object and optionally contains model and messages fields.

3. Prompt-Injection Guard: A singleton security guard from src/middleware/promptInjectionGuard.ts inspects the payload for injection attempts before forwarding to downstream handlers.

4. Model Alias Resolution: resolveModelAliasWithSeedFallbackOnBody rewrites any model aliases to their canonical provider identifiers.

5. Provider Availability Checks: assertRuntimeModelProviderAvailable and assertCommonChatGptWebModelAvailable ensure the targeted provider is active and the requested model is accessible.

6. Streaming Decision: The route inspects the Accept header and the optional stream: true body flag to determine whether to return a JSON object or establish an SSE stream.

7. Chat Handling: handleChat (located in open-sse/handlers/chat.ts) performs the actual provider dispatch, response translation via open-sse/translator/index.ts, and SSE keep-alive framing using OPENAI_KEEPALIVE_FRAME from open-sse/utils/earlyStreamKeepalive.ts.

Core Implementation Files

The endpoint behavior is distributed across several key modules:

• src/app/api/v1/chat/completions/route.ts: Implements the POST endpoint, request validation, security guards, and streaming logic decisions.

• open-sse/handlers/chat.ts: Core handler that forwards validated requests to the appropriate provider executor and manages response translation.

• open-sse/translator/index.ts: Loads provider-specific translators that convert heterogeneous provider payloads into the standardized OpenAI response format.

• open-sse/utils/earlyStreamKeepalive.ts: Supplies SSE keep-alive frames (OPENAI_KEEPALIVE_FRAME) that mimic OpenAI's streaming behavior to prevent connection timeouts.

• src/middleware/promptInjectionGuard.ts: Provides the security middleware used to sanitize inputs before they reach provider executors.

Usage Examples

Using curl

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Hello, world!"}]
      }'

Using Node.js with fetch

import fetch from "node-fetch";

const resp = await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Hello, world!" }],
    stream: true,               // Optional: enable SSE streaming
  }),
});

if (resp.ok) {
  const data = await resp.json();
  console.log(data);
}

Using the OpenAI Client Library

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:20128",   // OmniRoute base URL
  apiKey: "any-key-accepted-by-OmniRoute", // API key handling is optional
});

const chat = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello, world!" }],
});

console.log(chat);

Summary

• The OpenAI-compatible endpoint for OmniRoute is /v1/chat/completions (POST).
• The route implementation lives in src/app/api/v1/chat/completions/route.ts.
• Requests pass through validation, prompt-injection guards, model alias resolution, and provider availability checks before dispatch.
• The endpoint supports both standard JSON responses and Server-Sent Events (SSE) streaming, controlled via the stream parameter or Accept header.
• handleChat in open-sse/handlers/chat.ts manages provider-specific execution and response translation, ensuring drop-in compatibility with existing OpenAI clients.

Frequently Asked Questions

What is the exact URL path for OmniRoute's OpenAI-compatible API?

Clients should send POST requests to /v1/chat/completions relative to the OmniRoute base URL (for example, http://localhost:20128/v1/chat/completions). This path mirrors OpenAI's official API structure exactly, requiring no path modifications when switching from OpenAI's servers to OmniRoute.

Does OmniRoute support streaming responses like OpenAI's API?

Yes. When a request includes stream: true or sets an appropriate Accept header, the route delegates to handleChat in open-sse/handlers/chat.ts, which returns Server-Sent Events (SSE). The implementation uses OPENAI_KEEPALIVE_FRAME from open-sse/utils/earlyStreamKeepalive.ts to maintain connections during provider processing, matching OpenAI's streaming behavior precisely.

How does OmniRoute handle model specification in requests?

The endpoint uses resolveModelAliasWithSeedFallbackOnBody to rewrite model aliases to their canonical identifiers. Subsequently, assertRuntimeModelProviderAvailable and assertCommonChatGptWebModelAvailable verify that the resolved provider is active and the specific model is available before executing the request.

What security measures protect the chat completions endpoint?

A singleton prompt-injection guard defined in src/middleware/promptInjectionGuard.ts inspects every payload immediately after Zod schema validation. This guard screens for injection attempts before the request proceeds to provider dispatch, adding a critical security layer to the standard OpenAI-compatible flow.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}