How to Integrate CLI Tools with OmniRoute's Automatic Provider Detection

OmniRoute's CLI automatically discovers the best-fit LLM provider for each request using its AUTO-COMBO scoring system, requiring no hard-coded provider selection in your scripts or terminal commands.

The OmniRoute open-source routing engine (diegosouzapw/OmniRoute) ships with a lightweight command-line interface that transparently handles provider selection. Instead of manually choosing between OpenAI, Anthropic, or local models, you can write automation scripts that adapt dynamically to cost, latency, and availability. This guide explains how the auto-detection pipeline works and how to leverage it in your CLI workflows.

---

How Automatic Provider Detection Works in the CLI

When you run omniroute chat, the binary at bin/omniroute.mjs executes a five-stage pipeline. Understanding these stages helps you debug routing decisions and optimize your integration.

Stage 1: Environment Loading

The CLI first reads your global .env file or the generated .env from the repository root. This supplies API keys, OAuth tokens, and custom endpoint URLs to the runtime.

The loader is implemented in bin/cli/utils/environment.mjs and runs before any network calls, ensuring secrets are available to the routing layer.

Stage 2: Argument Parsing

The parser in bin/cli/utils/parseEnvValue.mjs handles flags like --model, --temperature, and --prompt. This is the critical decision point:

• If --model is supplied: The CLI forwards that name directly, bypassing auto-detection
• If no model is given: The CLI proceeds to automatic provider detection by leaving the provider field empty in the request payload

Stage 3: Request Construction

The CLI builds an OpenAI-compatible request payload matching the shape expected by open-sse/handlers/chatCore.ts. Key fields include:

{
  "prompt": "Your input text",
  "temperature": 0.7,
  "max_tokens": 256,
  "provider": ""  // Empty string triggers auto-detection
}

Stage 4: Auto-Combo Scoring and Selection

The core detection logic lives in open-sse/services/combo.ts. When the internal client (src/lib/a2a/clients/omniClient.ts) POSTs to /v1/chat/completions, the router evaluates the AUTO-COMBO strategy documented in docs/routing/AUTO-COMBO.md.

This strategy scores each candidate provider across four dimensions:

• Latency – Recent response time measurements
• Cost – Per-token pricing for the requested model class
• Quota – Remaining rate limit headroom
• Model availability – Whether the provider serves the needed capabilities

The highest-scoring provider is selected automatically. No intervention required.

Stage 5: Response Streaming

The CLI streams Server-Sent Events (SSE) back to your terminal by default, or returns a JSON blob if you pass --stream=false. Errors are sanitized through open-sse/utils/error.ts before display.

---

CLI Commands for Automatic Provider Detection

Basic Auto-Detected Prompt

The simplest invocation lets OmniRoute choose everything:

omniroute chat "Explain the theory of relativity in three sentences"

The routing layer evaluates all registered providers and picks the optimal target.

Bypassing Auto-Detection with Explicit Model Selection

To force a specific provider and model, use the --model flag:

omniroute chat --model=gpt-4o "Write a short poem about sunrise"

This skips the AUTO-COMBO scoring and routes directly to the provider hosting gpt-4o.

Adjusting Generation Parameters

Control sampling behavior while still using automatic provider selection:

omniroute chat \
  --temperature=0.7 \
  --max_tokens=256 \
  "Summarize the latest news about quantum computing"

The router still selects the provider, but applies your parameters to the generation.

---

Integrating the CLI into Scripts and Automation

Node.js Integration

Since the CLI returns structured output, you can wrap it in any scripting environment:

import { execSync } from "child_process";

const prompt = "List three ways to improve developer productivity.";
const result = execSync(`omniroute chat "${prompt}"`, { encoding: "utf-8" });
console.log("OmniRoute answer:", result);

The omniClient.ts implementation handles authentication headers internally, so your script needs no credential management logic.

Shell Pipeline Integration

Chain OmniRoute with standard Unix tools:

cat article.txt | xargs -I {} omniroute chat "Summarize: {}" > summary.txt

Auto-detection runs per request, so a long-running pipeline can route different calls to different providers based on real-time conditions.

---

Extending Auto-Detection to New Providers

The CLI requires zero code changes to recognize new providers. To add support:

1. Register the provider in open-sse/config/providerRegistry.ts
2. Optionally implement a custom executor in open-sse/executors/

On the next CLI invocation, the AUTO-COMBO scorer in combo.ts automatically includes the new provider in its evaluation. This applies to:

• API-key providers (OpenAI, Anthropic, etc.)
• OAuth-based services (Google, Microsoft)
• Locally-hosted models (Ollama, vLLM, llama.cpp)

---

Configuration Files and Key Locations

Path	Purpose
bin/omniroute.mjs	CLI entry point
bin/cli/utils/environment.mjs	Loads .env configuration and secrets
bin/cli/utils/parseEnvValue.mjs	Parses flags, determines auto-detect mode
open-sse/config/providerRegistry.ts	Central registry of all providers and models
open-sse/services/combo.ts	AUTO-COMBO scoring and selection engine
open-sse/handlers/chatCore.ts	Core request handler for OpenAI-compatible payloads
src/lib/a2a/clients/omniClient.ts	Internal HTTP client used by the CLI

---

Summary

• OmniRoute CLI automatically selects LLM providers using the same AUTO-COMBO scoring engine that powers its HTTP API
• Leave the --model flag empty to trigger detection; supply it to bypass and force a specific provider
• The detection pipeline runs through environment.mjs → parseEnvValue.mjs → omniClient.ts → combo.ts → chatCore.ts
• New providers are picked up automatically after registry addition—no CLI rebuild required
• Integrate via shell scripts, Node.js execSync, or any subprocess wrapper without managing provider-specific credentials

---

Frequently Asked Questions

What happens if no provider passes the AUTO-COMBO scoring threshold?

The router returns a structured error from open-sse/utils/error.ts indicating no viable provider was found. This typically occurs when all registered providers exceed rate limits or lack quota. Check your .env configuration and provider registry entries.

Can I prioritize cost over latency in auto-detection?

As of version v3.8.50, the AUTO-COMBO weights are fixed in combo.ts. The scoring balances all four dimensions equally. Future releases may expose weight configuration through CLI flags or environment variables.

Does automatic provider detection add latency to each request?

The scoring evaluation adds negligible overhead (sub-millisecond) because it operates on cached provider metrics maintained by the routing layer. The actual provider selection completes before the outbound HTTP connection opens.

How do I debug which provider was selected for a request?

Run with verbose logging enabled (check bin/omniroute.mjs for --verbose or DEBUG environment support). The router logs the selected provider and its score components to stderr before streaming the response.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}