OmniRoute OpenAI-Compatible Endpoint: Complete Implementation Guide

OmniRoute exposes a fully OpenAI-compatible REST API through the /v1/chat/completions endpoint, enabling drop-in replacement for OpenAI clients without code modifications.

OmniRoute's primary OpenAI-compatible endpoint is the /v1/chat/completions route. This endpoint mirrors OpenAI's Chat Completion API specification, accepting identical request payloads and returning responses in the same JSON schema. Whether you're using curl, standard HTTP clients, or the official OpenAI SDK, you can point your requests at OmniRoute and expect seamless compatibility.

The /v1/chat/completions Endpoint

The core entry point lives at src/app/api/v1/chat/completions/route.ts. This file implements a complete pipeline that processes incoming requests through seven distinct stages before dispatching to backend providers.

Request Processing Pipeline

When you POST to /v1/chat/completions, OmniRoute executes the following flow:

1. CORS handling — handleCorsOptions in src/shared/utils/cors.ts processes OPTIONS pre-flight requests and sets appropriate headers.
2. Request shape validation — The chatCompletionsRouteShapeSchema Zod schema validates that the body is an object with optional model and messages fields.
3. Prompt injection guard — The singleton guard from src/middleware/promptInjectionGuard.ts inspects payloads for injection attempts.
4. Model alias resolution — resolveModelAliasWithSeedFallbackOnBody rewrites any configured model aliases to actual provider model names.
5. Provider availability checks — assertRuntimeModelProviderAvailable and assertCommonChatGptWebModelAvailable verify the target provider is active.
6. Streaming decision — The Accept header and optional stream: true flag determine SSE streaming versus single JSON response.
7. Chat handling — handleChat in open-sse/handlers/chat.ts dispatches to providers, translates responses, and manages SSE keep-alive framing.

Streaming and Non-Streaming Modes

OmniRoute supports both response modes identically to OpenAI:

• Non-streaming: Returns a complete JSON payload with the full completion.
• Streaming: Returns Server-Sent Events (SSE) with incremental token deltas.

The streaming implementation uses OPENAI_KEEPALIVE_FRAME from open-sse/utils/earlyStreamKeepalive.ts to maintain connection health during provider latency.

Code Examples for the OpenAI-Compatible Endpoint

Using curl

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Hello, world!"}]
      }'

Using Node.js fetch

import fetch from "node-fetch";

const resp = await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Hello, world!" }],
    stream: true,               // Enable Server-Sent Events streaming
  }),
});

// Handle streaming or JSON response based on stream flag
if (resp.ok && !resp.headers.get("content-type")?.includes("text/event-stream")) {
  const data = await resp.json();
  console.log(data);
}

Using the Official OpenAI SDK

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:20128",   // OmniRoute base URL replaces api.openai.com
  apiKey: "any-key-accepted-by-OmniRoute", // API key validation is configurable
});

const chat = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello, world!" }],
});
console.log(chat.choices[0].message);

Key Implementation Files

File	Role
src/app/api/v1/chat/completions/route.ts	Main route handler implementing POST endpoint, validation, guards, and streaming logic
open-sse/handlers/chat.ts	Core dispatch handler that routes to provider executors and manages response translation
open-sse/translator/index.ts	Translator registry that loads provider-specific payload converters to OpenAI format
open-sse/utils/earlyStreamKeepalive.ts	SSE keep-alive frame definitions (OPENAI_KEEPALIVE_FRAME) matching OpenAI's streaming behavior
src/shared/utils/cors.ts	CORS pre-flight and response header management
src/middleware/promptInjectionGuard.ts	Security middleware for prompt injection detection

Summary

• OmniRoute's OpenAI-compatible endpoint is /v1/chat/completions, hosted at your configured base URL.
• Full API compatibility means existing OpenAI SDK code works with only a baseURL change.
• Request pipeline includes validation, security checks, alias resolution, and provider availability verification before dispatch.
• Streaming support via SSE uses OpenAI-identical keep-alive frames for reliable real-time responses.
• Core files in src/app/api/v1/chat/completions/ and open-sse/ directories implement the complete translation layer.

Frequently Asked Questions

What URL path should I use for OmniRoute's OpenAI-compatible API?

Use /v1/chat/completions appended to your OmniRoute base URL. For local development with default settings, this is http://localhost:20128/v1/chat/completions. The path matches OpenAI's API exactly, so only the domain and port need to change in client configurations.

Does OmniRoute support streaming responses like OpenAI?

Yes. OmniRoute's OpenAI-compatible endpoint supports both streaming and non-streaming modes. Set stream: true in your request body to receive Server-Sent Events with incremental deltas. The implementation in open-sse/utils/earlyStreamKeepalive.ts uses OPENAI_KEEPALIVE_FRAME to maintain OpenAI-identical streaming behavior during provider latency.

Can I use the official OpenAI Python or Node.js SDK with OmniRoute?

Absolutely. Configure the SDK with OmniRoute's baseURL and any API key (OmniRoute's key validation is optional). The SDK's chat.completions.create() method works unchanged because OmniRoute's endpoint returns identical response schemas for both streaming and non-streaming requests.

How does OmniRoute handle model names in requests?

OmniRoute resolves model aliases through resolveModelAliasWithSeedFallbackOnBody before provider dispatch. You can configure aliases in OmniRoute's settings to map friendly names like "gpt-4o-mini" to specific provider model identifiers, or pass provider-native model names directly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}