How to Use the Chat Completions Endpoint in OmniRoute: A Complete API Guide

The Chat Completions endpoint in OmniRoute is a drop-in replacement for OpenAI's API, available at POST /api/v1/chat/completions, routing requests through validation, provider selection, and resilience layers before returning OpenAI-compatible responses.

OmniRoute (as implemented in diegosouzapw/OmniRoute) exposes a single OpenAI-compatible HTTP API that normalizes requests across multiple LLM providers. Because the Chat Completions endpoint in OmniRoute mirrors OpenAI's specification exactly, existing clients require zero configuration changes to route traffic through the gateway. The implementation handles everything from prompt injection guards to provider-specific payload translation.

How the Chat Completions Pipeline Works

Requests flow through a strictly defined seven-stage pipeline before reaching upstream providers.

URL Normalization and Routing

The journey begins in src/server/authz/classify.ts, where the gateway normalizes incoming URLs. The alias /chat/completions maps to the canonical /api/v1/chat/completions, ensuring compatibility with clients that use OpenAI's default path structure. The API surface is registered in src/app/api/v1/vscode/raw/[token]/v1/chat/completions/route.ts, while src/shared/constants/endpointCategories.ts classifies the endpoint type to apply appropriate middleware chains.

Content Validation Middleware

Before parsing the body, src/shared/middleware/requireJsonContentType.ts enforces strict Content-Type headers. Non-JSON payloads are rejected immediately, protecting downstream services from malformed requests.

Body Admission and Security

The src/shared/middleware/chatBodyAdmission.ts module handles request body parsing once, applying the prompt-injection guard and size limits enforced by src/shared/middleware/bodySizeGuard.ts. This single-pass parsing prevents duplicate computation and blocks oversized payloads before they consume provider tokens.

Core Chat Handler Logic

The src/sse/handlers/chat.ts file contains the primary business logic. It validates the requested model against the catalog, rejects image-generation models (which require different endpoints), and delegates provider-specific formatting to helper functions in src/sse/handlers/chatHelpers.ts.

Provider Translation Layer

OmniRoute dynamically selects an executor from src/open-sse/executors/ and a translator from src/open-sse/translator/ based on the target provider. These components convert the OpenAI-style payload into provider-specific formats, handling differences in message structure, parameter naming, and authentication schemes.

Resilience Mechanisms

Provider-level circuit breakers, connection cooldowns, and model lockouts implemented in src/open-sse/services/* protect the Chat Completions endpoint from transient upstream failures. If a provider returns errors or timeouts, OmniRoute automatically fails over to alternatives without client intervention.

Response Streaming

Finally, upstream streams are translated back into the OpenAI Chat Completion format. The gateway supports both Server-Sent Events (SSE) for streaming and assembled JSON responses, depending on the stream parameter in the request.

Making Requests to the Chat Completions Endpoint

Standard HTTP Request

Send a synchronous request using any HTTP client. The endpoint accepts the exact same payload structure as OpenAI's official API.

Using curl:

curl -X POST https://localhost:20128/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMNIROUTE_API_KEY" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "Tell me a joke."}
        ],
        "temperature": 0.7
      }'

Using JavaScript fetch:

const resp = await fetch('https://localhost:20128/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OMNIROUTE_API_KEY}`
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [
      {role: 'system', content: 'You are a helpful assistant.'},
      {role: 'user', content: 'Tell me a joke.'}
    ],
    temperature: 0.7
  })
});
const data = await resp.json();
console.log(data);

Streaming with Server-Sent Events

For real-time token streaming, set stream: true in the request body. The connection returns SSE chunks compatible with OpenAI's streaming format.

Using Node.js streams:

import { createReadStream } from 'node:stream';
import fetch from 'node-fetch';

const response = await fetch('https://localhost:20128/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OMNIROUTE_API_KEY}`
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{role: 'user', content: 'Explain quantum entanglement.'}],
    stream: true
  })
});

for await (const chunk of response.body) {
  process.stdout.write(chunk);
}

Request Validation and Security

The Chat Completions endpoint implements defense-in-depth through middleware chains defined in src/shared/middleware/.

Prompt Injection Guard: The chatBodyAdmission.ts middleware scans incoming messages for known prompt injection patterns before forwarding to providers.

Size Limits: The bodySizeGuard.ts enforces maximum payload sizes, preventing denial-of-service attacks through excessively large context windows.

Authorization: All requests require a Bearer token in the Authorization header, validated against the API key store before routing begins.

Provider Selection and Translation

OmniRoute automatically discovers available models through the /api/v1/models catalog endpoint. When you specify a model in your request, the system:

1. Locates the provider configuration in src/shared/constants/providers/apikey/*
2. Selects the appropriate executor from src/open-sse/executors/
3. Applies the translator from src/open-sse/translator/ to convert the payload
4. Forwards the transformed request to the provider's native /v1/chat/completions surface

This architecture allows the Chat Completions endpoint to support any provider exposing an OpenAI-compatible interface without code changes to your client.

Resilience and Error Handling

The gateway protects against cascading failures through several mechanisms implemented in src/open-sse/services/*:

Circuit Breakers: Temporarily disable providers returning consecutive errors. Connection Cooldowns: Rate-limit connections to providers showing latency spikes. Model Lockouts: Remove specific model versions from rotation when they returning persistent failures.

These mechanisms ensure that the Chat Completions endpoint maintains high availability even when individual upstream providers experience outages.

Summary

• The Chat Completions endpoint in OmniRoute is accessible at POST /api/v1/chat/completions and accepts OpenAI-compatible request bodies.
• Requests pass through seven pipeline stages: routing, CORS validation, body admission, chat handling, provider translation, resilience checks, and response streaming.
• Key implementation files include src/sse/handlers/chat.ts for core logic and src/server/authz/classify.ts for URL normalization.
• The endpoint supports both standard JSON responses and Server-Sent Events (SSE) streaming.
• Automatic provider discovery uses the model catalog at /api/v1/models and configurations in src/shared/constants/providers/apikey/*.

Frequently Asked Questions

What URL should I use to call the Chat Completions endpoint?

Use POST /api/v1/chat/completions as the canonical endpoint. The gateway also accepts /chat/completions (aliased in src/server/authz/classify.ts) for compatibility with OpenAI client defaults. Both paths route to the same handler in src/sse/handlers/chat.ts.

Does OmniRoute support streaming responses like OpenAI?

Yes. Set "stream": true in your request body to enable Server-Sent Events (SSE). The src/open-sse/translator/ layer transforms provider-specific streaming formats back into OpenAI-compatible SSE chunks, ensuring clients like LangChain or the OpenAI SDK work without modification.

How does OmniRoute handle provider failures when calling the Chat Completions endpoint?

The system implements circuit breakers, connection cooldowns, and model lockouts in src/open-sse/services/. If a provider fails, the gateway automatically routes to healthy alternatives listed in src/shared/constants/providers/apikey/* without returning errors to the client.

Can I use image generation models through the Chat Completions endpoint?

No. The src/sse/handlers/chat.ts handler explicitly validates models and rejects image-generation models, which require different payload structures. Use the dedicated image generation endpoints for providers like DALL-E, or check the model catalog at /api/v1/models for supported completion models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}