OmniRoute Basic Usage Examples: CLI, REST API, and Node SDK Quick Start

OmniRoute acts as a unified AI proxy that automatically routes LLM requests to optimal providers using a 16-factor scoring algorithm, exposing three interchangeable interfaces—a bundled CLI, OpenAI-compatible REST endpoints, and a generated Node SDK—all built on a Next.js App Router architecture.

The open-source OmniRoute repository (diegosouzapw/OmniRoute) provides a production-ready routing layer for 300+ LLM providers. Whether you are testing locally or deploying at scale, you can interact with the unified API through command-line tools, direct HTTP requests, or programmatic TypeScript clients without changing your underlying prompt logic.

Local Server Setup and Installation

Before executing any OmniRoute basic usage examples, you must bootstrap the Next.js application that hosts the streaming engine.

npm install
cp .env.example .env
npm run dev

This initializes the development server on port 20128 (default), loading the provider executors and resilience layers defined in src/lib/resilience/ and src/lib/db/.

CLI Usage Examples

The bundled command-line interface in bin/cli/ provides shortcuts for rapid local testing.

Execute a chat completion directly from your terminal:

omniroute chat "Explain quantum entanglement in simple terms."

The CLI wraps the core server and automatically handles authentication via environment variables, routing your prompt through the same handleChatCore() logic found in open-sse/handlers/chatCore.ts that manages caching and rate-limiting.

HTTP API Usage Examples

Send standard OpenAI-compatible POST requests to the Next.js App Router endpoints.

Chat Completions

Target the route defined in src/app/api/v1/chat/completions/route.ts:

curl -X POST http://localhost:20128/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hello"}]}'

Embeddings

Target the route defined in src/app/api/v1/embeddings/route.ts:

curl -X POST http://localhost:20128/api/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"text-embedding-3-large","input":"machine learning"}'

Each request passes through Zod validation, optional authentication, and policy checks before translation and dispatch.

Node SDK Integration

Import the generated client from @omniroute/opencode-provider to interact with OmniRoute using TypeScript.

Basic Chat Completion

import { OmniRouteClient } from "@omniroute/client";

const client = new OmniRouteClient({ apiKey: process.env.OMNIROUTE_API_KEY });

const resp = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "What is the capital of France?" }]
});
console.log(resp.choices[0].message.content);

The SDK mirrors the OpenAI API surface while handling authentication, rate-limit back-off, and auto-fallback internally.

Combo Routing with Auto-Selection

Trigger the auto-combo strategy to let the router choose the provider:

await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Summarize today's news" }]
});

The Request Pipeline Architecture

Understanding the internal flow helps debug routing decisions and optimize performance.

Entry Point Validation

Requests enter through src/app/api/v1/ routes, where Zod schemas validate payloads before passing them to the streaming engine.

Streaming Engine and Resilience

The handleChatCore() function in open-sse/handlers/chatCore.ts orchestrates the request lifecycle, interacting with the SQLite-based persistence layer in src/lib/db/ and applying circuit-breaker logic from src/lib/resilience/ to prevent cascading failures.

Provider Selection Algorithm

The combo-routing layer in open-sse/services/combo.ts implements 19 distinct routing strategies, including the 16-factor scoring algorithm that evaluates 300+ providers to determine the optimal target for each request.

Configuration and Guardrails

Customize behavior without modifying code.

Disable PII redaction (opt-in) by setting the environment variable referenced in src/lib/guardrails/piiMasker.ts:

export OMNIROUTE_PII_REDACTION_ENABLED=false

Summary

• OmniRoute provides three interchangeable interfaces: CLI (bin/cli/), REST API (src/app/api/v1/), and Node SDK (@omniroute/opencode-provider).
• All entry points route through handleChatCore() in open-sse/handlers/chatCore.ts and the combo-routing algorithm in open-sse/services/combo.ts.
• The system supports 300+ providers with automatic failover, circuit breakers in src/lib/resilience/, and SQLite-backed state management in src/lib/db/.
• Requests follow a validated pipeline: Zod schema checking, policy enforcement, provider translation, dispatch, and response translation back to OpenAI-compatible formats.

Frequently Asked Questions

What port does OmniRoute use for local development?

By default, the Next.js development server listens on port 20128. You can override this by setting the PORT environment variable before running npm run dev.

How does OmniRoute choose which LLM provider to use?

The router applies a 16-factor scoring algorithm implemented in open-sse/services/combo.ts through the "auto-combo" strategy. When you specify "model": "auto", the system evaluates latency, cost, and reliability metrics across 300+ providers to select the optimal endpoint.

Can I use OmniRoute as a drop-in replacement for the OpenAI SDK?

Yes. The Node SDK generated in @omniroute/opencode-provider exposes identical method signatures to the OpenAI client. All chat and embedding methods accept standard OpenAI payload structures, making migration a simple import swap.

Where does OmniRoute store provider credentials and circuit-breaker state?

The system uses SQLite databases located in src/lib/db/ to persist provider credentials, feature flags, and circuit-breaker state, ensuring resilience patterns survive server restarts without requiring external database infrastructure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}