How to Add a Custom OpenAI-Compatible Provider to OmniRoute

You add a custom OpenAI-compatible provider to OmniRoute by creating a RegistryEntry module in open-sse/config/providers/registry/ that sets format: "openai", registering it in the exported REGISTRY map, and allowing the default executor to handle request construction and authentication automatically.

OmniRoute discovers every LLM endpoint through its centralized Provider Registry (open-sse/config/providerRegistry.ts). Each registry entry describes the provider's API format, authentication method, base URLs, and model catalogue, enabling the routing engine to treat any OpenAI-compatible service as a first-class citizen for combo routing and fallback logic.

Understanding the Provider Registry Architecture

The Provider Registry serves as the single source of truth for all LLM endpoints in OmniRoute. Located at open-sse/config/providerRegistry.ts, this system maintains a REGISTRY map that associates provider IDs with their configuration entries. Lookup helpers such as getRegistryEntry and getRegisteredProviders consume this registry to resolve routing decisions, validate authentication schemes, and determine if a provider uses an authoritative live catalog.

Each provider entry follows the RegistryEntry interface, which specifies the API format—such as "openai" for OpenAI-compatible endpoints—the base URL, chat completion path, headers, and available models. This declarative approach means you can integrate a new service without modifying the core routing logic, provided it conforms to the OpenAI chat-completions specification.

Step 1: Create the Registry Module

Create a new folder under open-sse/config/providers/registry/ to house your provider definition. For example, create my-custom/ containing an index.ts file that exports a RegistryEntry object. This entry must declare format: "openai" to signal that the endpoint follows the OpenAI chat-completions API structure.

// open-sse/config/providers/registry/my-custom/index.ts
import type { RegistryEntry } from "../shared.ts";

export const myCustomProvider: RegistryEntry = {
  id: "my-custom",
  format: "openai",                     // Signals OpenAI-compatible format
  authType: "apikey",                   // Uses Bearer token in Authorization header
  baseUrl: "https://api.my-custom.com", // Service base URL
  chatPath: "/v1/chat/completions",     // Standard OpenAI endpoint path
  headers: { "Content-Type": "application/json" },
  requestDefaults: { timeoutMs: 30_000 },
  models: [
    {
      id: "gpt-custom-1",
      contextLength: 8192,
      // Optional: capabilities, pricing, etc.
    },
  ],
};

Step 2: Register in the Central Index

Expose your module by importing it into open-sse/config/providers/registry/index.ts and adding it to the exported REGISTRY record. This registration makes the provider available to the routing system and model discovery endpoints.

// open-sse/config/providers/registry/index.ts
import { myCustomProvider } from "./my-custom/index.ts";

export const REGISTRY: Record<string, RegistryEntry> = {
  // ...existing providers
  "my-custom": myCustomProvider,
};

Step 3: Leverage the Default Executor

Because your provider uses the standard OpenAI format, you do not need to write custom execution logic. The default executor at open-sse/executors/default.ts automatically constructs requests, attaches the Bearer token from the authType: "apikey" configuration, and forwards responses unchanged. This executor handles streaming and non-streaming completions according to the OpenAI specification.

You only need a custom executor if your provider exhibits non-standard behavior, such as unique streaming quirks or proprietary authentication flows. For standard OpenAI-compatible endpoints, the default implementation suffices.

Step 4: Define Available Models

Populate the models array in your registry entry to expose the provider's capabilities via the /v1/models route. Each model object includes an id, contextLength, and optional metadata such as capabilities or pricing. This metadata enables combo routing to select appropriate models based on context window requirements or cost constraints.

models: [
  {
    id: "gpt-custom-1",
    contextLength: 8192,
    pricing: { prompt: 0.001, completion: 0.002 }
  },
  {
    id: "gpt-custom-2",
    contextLength: 32768,
  }
]

Step 5: Write Unit Tests

Validate your integration by copying the test pattern from existing OpenAI-style provider tests, such as tests/unit/openai-style-providers-4239-4155-3841.test.ts. Your tests should verify that the registry entry exists, that format equals "openai", and that baseUrl and chatPath resolve to valid URLs. This satisfies OmniRoute's mandatory test-coverage requirements and prevents regression.

Example test assertions:

• Verify REGISTRY["my-custom"] is defined
• Assert REGISTRY["my-custom"].format === "openai"
• Check that new URL(REGISTRY["my-custom"].chatPath, REGISTRY["my-custom"].baseUrl) is valid

Step 6: Run the Full Test Suite

Execute npm run test:unit and npm run test:vitest to ensure your provider does not break combo routing, resilience layers, or translation mechanisms. All existing tests must pass before your provider can be considered safely integrated.

Step 7: Document the Provider

Add a concise entry to the auto-generated provider reference at docs/reference/PROVIDER_REFERENCE.md or your internal knowledge base. Link directly to your registry source file (open-sse/config/providers/registry/my-custom/index.ts) so administrators can review configuration details.

Critical Source Files

File	Purpose
open-sse/config/providerRegistry.ts	Defines the RegistryEntry interface and lookup helpers like getRegistryEntry and providerUsesAuthoritativeLiveCatalog.
open-sse/config/providers/registry/index.ts	Central export point that constructs the REGISTRY map imported by the routing engine.
open-sse/config/providers/registry/openai/index.ts	Reference implementation showing the exact shape required for OpenAI-compatible entries.
open-sse/executors/default.ts	Generic executor that handles request building and authentication for format: "openai" providers.

Summary

• Create a new folder under open-sse/config/providers/registry/ containing an index.ts that exports a RegistryEntry with format: "openai".
• Register the provider in open-sse/config/providers/registry/index.ts by importing it and adding it to the REGISTRY map.
• Rely on open-sse/executors/default.ts to handle request construction and Bearer token authentication automatically.
• Define models in the entry's models array to enable discovery via /v1/models and combo routing.
• Test using the pattern in tests/unit/openai-style-providers-4239-4155-3841.test.ts and run npm run test:unit to verify integration.
• Document the provider in docs/reference/PROVIDER_REFERENCE.md with a link to the source configuration.

Frequently Asked Questions

What authentication methods does OmniRoute support for custom providers?

OmniRoute supports multiple authentication schemes through the authType field in the registry entry. For OpenAI-compatible providers, you typically set authType: "apikey", which instructs the default executor to attach a Bearer token in the Authorization header. Other supported types may include custom header-based authentication, depending on the executor implementation.

Do I need to write custom code to handle streaming responses?

No. If your provider follows the standard OpenAI chat-completions format and you set format: "openai" in the registry entry, the default executor at open-sse/executors/default.ts automatically handles both streaming and non-streaming responses. It manages the Server-Sent Events (SSE) format and forwards the stream unchanged to the client.

How does OmniRoute discover the models available from my custom provider?

OmniRoute exposes models through the /v1/models endpoint based on the models array defined in your RegistryEntry. Each object in this array must include at least an id and contextLength. The routing system uses this metadata for combo routing decisions, matching requests to appropriate models based on context window requirements and other capabilities.

Can I use environment variables for the base URL or API keys?

Yes. While the registry entry defines static configuration in open-sse/config/providers/registry/my-custom/index.ts, you can import environment variables at the module level to set baseUrl or other configuration values. The registry file is standard TypeScript, allowing you to externalize sensitive values or deployment-specific URLs using your runtime's environment variable API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}