How OmniRoute Handles OpenAI API Compatibility: A Complete Technical Guide

OmniRoute achieves OpenAI API compatibility through a bidirectional translation layer that converts OpenAI-formatted requests into provider-native formats and transforms provider responses back into the OpenAI schema, enabling drop-in SDK replacement.

OmniRoute acts as a transparent proxy between OpenAI SDK clients and diverse LLM providers. The system implements OpenAI API compatibility through a cohesive pipeline of request translators, provider executors, and response translators—all orchestrated in open-sse/handlers/chatCore.ts. This architecture allows developers to point existing OpenAI code at an OmniRoute endpoint without modification while the router intelligently distributes traffic across OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, and other supported backends.

How the Compatibility Pipeline Works

The request lifecycle follows three distinct stages, each implemented by dedicated modules in the codebase.

Stage 1: OpenAI Request Translation

Incoming requests to /v1/chat/completions or /v1/completions are parsed and converted from OpenAI's JSON schema into OmniRoute's internal message representation. The translation modules live in open-sse/translator/request/:

• openai-to-claude.ts – Transforms OpenAI message arrays into Anthropic's Claude-native format
• openai-to-gemini.ts – Adapts requests for Google's Gemini API structure
• openai-responses/toResponses.ts – Handles the specialized "openai-responses" protocol for list-style response bodies

The system detects the appropriate translator through filterToOpenAIFormat in open-sse/translator/helpers/openaiHelper.ts. This helper examines incoming payloads to determine whether standard OpenAI format or the newer OpenAI-Responses format applies.

Stage 2: Provider Execution

Once translated, requests route through provider-specific executors in open-sse/executors/:

• DefaultExecutor (open-sse/executors/default.ts) – Handles "openai-compatible" providers (those exposing OpenAI-compatible HTTP endpoints). It selects base URLs like https://api.openai.com/v1 by default and forwards translated requests verbatim.
• Specialized executors – Custom implementations for providers requiring unique authentication, rate limiting, or request signing.

At open-sse/executors/zed-hosted.ts line 90, the executor checks each model's targetFormat property (e.g., "openai" or "openai-responses"). This model-level override determines which translator path executes, enabling fine-grained routing decisions per deployment.

Stage 3: OpenAI Response Translation

Provider responses undergo reverse translation in open-sse/translator/response/:

• openai-to-claude.ts – Maps Claude response fields (content, role, stop_reason) onto OpenAI's choices array structure
• openai-responses.ts – Normalizes the Responses API format (including streaming data: {...} events) back to classic Chat Completion shape

The final payload matches OpenAI's official schema exactly, preserving field names, nesting, and data types expected by the client SDK.

Key Compatibility Features Explained

Native OpenAI-Compatible Provider Support

Providers configured with a provider string starting with openai-compatible- automatically route through DefaultExecutor. This includes Azure OpenAI, local deployments (vLLM, Ollama), and third-party APIs mimicking OpenAI's interface. The executor maintains fallback logic to the official OpenAI endpoint when specific routing rules don't match.

OpenAI-Responses Protocol Support

The openai-responses translator adds support for OpenAI's newer Responses API, which differs from traditional Chat Completions in several ways:

• Returns list-style output structures
• Emits distinct streaming chunk formats
• Requires specialized tool-call synthesis

open-sse/translator/response/openai-responses.ts bridges these differences, ensuring older clients receive compatible responses.

Tool-Call Mapping and Normalization

Custom tool names present a compatibility challenge when providers use different naming conventions. OmniRoute resolves this through:

• collectCustomToolNamesForSourceFormat (invoked in chatCore.ts) – Captures tool definitions from the original OpenAI-format request
• Response-time remapping – Restores OpenAI-compliant tool-call identifiers before returning to the client

This preserves tool/function calling semantics across provider boundaries without client-side changes.

Streaming (SSE) Handling

Server-sent events from upstream providers are wrapped and re-emitted through open-sse/translator/helpers/openaiHelper.ts. The helper ensures:

• Proper data: {...} frame prefixes
• Consistent event boundaries
• Error propagation in OpenAI-compatible format

Clients using stream: true receive indistinguishable behavior from direct OpenAI API access.

Practical Implementation Examples

Example 1: Drop-In SDK Replacement

Point any existing OpenAI SDK project at OmniRoute with two configuration changes:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://my-router.example.com/v1",  // OmniRoute endpoint
  apiKey: "omniroute-key-123",                  // OmniRoute API key
});

const chat = await client.chat.completions.create({
  model: "gpt-4o-mini",  // OmniRoute maps to configured provider
  messages: [{ role: "user", content: "Explain quantum tunnelling." }],
});

console.log(chat.choices[0].message.content);
// Response structure identical to direct OpenAI API

Example 2: Direct Translator Access (Debugging)

Inspect intermediate translation for troubleshooting:

import { openaiToClaudeRequest } from "@/open-sse/translator/request/openai-to-claude.ts";

const openaiPayload = {
  model: "gpt-4o",
  messages: [{ role: "user", content: "What is recursion?" }],
};

const { body, provider } = await openaiToClaudeRequest(
  "claude",
  openaiPayload,
  true   // isInternal flag: true when called within OmniRoute
);

// body contains Claude-compatible request JSON
// provider identifies the target execution path

Example 3: Azure OpenAI Via DefaultExecutor

Programmatic access to the executor layer:

import { DefaultExecutor } from "@/open-sse/executors/default.ts";

const exec = new DefaultExecutor("azure-openai");
const response = await exec.execute({
  model: "gpt-35-turbo",
  messages: [{ role: "user", content: "Summarise the latest news." }],
});

console.log(response);
// Already translated to OpenAI schema by response translator

Core Source Files for OpenAI Compatibility

Purpose	File Path
Claude request translation	open-sse/translator/request/openai-to-claude.ts
OpenAI-Responses request handling	open-sse/translator/request/openai-responses/toResponses.ts
Format detection and routing	open-sse/translator/helpers/openaiHelper.ts
Translator registry mapping	open-sse/translator/registry.ts
Default executor with fallback logic	open-sse/executors/default.ts
Main request handler orchestration	open-sse/handlers/chatCore.ts
OpenAI-Responses response normalization	open-sse/translator/response/openai-responses.ts
Model-level format overrides	open-sse/executors/zed-hosted.ts

Summary

• Bidirectional translation converts OpenAI requests to provider formats and back, transparent to clients
• Three-stage pipeline: request translation → provider execution → response translation
• DefaultExecutor handles OpenAI-compatible providers with automatic fallback to official endpoints
• targetFormat model property enables per-model routing decisions between standard and Responses API formats
• Tool calls and streaming receive full normalization through dedicated helper utilities
• Zero client changes required—existing OpenAI SDK code works with only baseURL modification

Frequently Asked Questions

Does OmniRoute support the latest OpenAI SDK versions?

Yes. As implemented in diegosouzapw/OmniRoute, the response translators maintain parity with OpenAI's Chat Completions schema, including choices, usage, and system_fingerprint fields. The openai-responses.ts translator specifically addresses newer Responses API differences. Both streaming and non-streaming modes return structurally identical responses to official OpenAI endpoints.

How does OmniRoute handle models not natively hosted by OpenAI?

The targetFormat property at the model level (open-sse/executors/zed-hosted.ts line 90) triggers appropriate translators. For example, a request with model: "claude-3-opus" routes through openai-to-claude.ts for request conversion, executes against Anthropic's API, then maps the response back through the corresponding response translator. The client always receives standard OpenAI-format output regardless of upstream provider.

What happens when a provider returns errors or rate limits?

DefaultExecutor in open-sse/executors/default.ts propagates HTTP status codes and error payloads through the response translation layer. Error responses are reformatted to match OpenAI's error schema with error.type, error.message, and error.param fields. This ensures client-side error handling—like retries on 429 rate limits or 500 server errors—functions identically to direct OpenAI API usage.

Can I use OmniRoute with local or self-hosted OpenAI-compatible servers?

Absolutely. Configure any provider with openai-compatible- prefix to route through DefaultExecutor. This supports vLLM, Ollama, text-generation-inference, and custom implementations exposing /v1/chat/completions endpoints. The executor forwards requests without additional translation when source and target formats align, minimizing latency for compatible deployments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}