OmniRoute Auto Routing Strategy: Zero-Config Dynamic Provider Selection

OmniRoute's auto routing strategy is a built-in Zero-Config Auto-Combo engine that dynamically constructs virtual provider combinations at request time by scoring available connections across 9 to 14 factors and selecting the optimal route based on configurable strategies.

The auto routing strategy eliminates manual provider configuration by automatically building ephemeral combinations from live connections. Implemented in the diegosouzapw/OmniRoute open-source repository, this feature intercepts requests bearing the auto/ prefix and routes them through a sophisticated scoring pipeline without persisting database entries.

How the Auto Routing Strategy Works

Prefix Detection and Virtual Combo Creation

When a request arrives at src/sse/handlers/chat.ts, the handler checks for the auto/ prefix in the model identifier. Upon detection, the system bypasses standard combo lookups and triggers the virtual combo factory in open-sse/services/autoCombo/virtualFactory.ts.

This module queries getProviderConnections to retrieve all active provider connections, filters for valid credentials, cross-references the provider registry for model availability, and generates a VirtualAutoComboCandidate for each valid tuple entirely in memory. No database rows are written during this process.

Multi-Factor Scoring Engine

The core scoring logic resides in open-sse/services/autoCombo/scoring.ts, which applies the DEFAULT_WEIGHTS configuration across nine standard factors—or fourteen when using extended scoring. Each candidate receives a weighted score based on metrics including quota availability, health status, cost inversion, and latency inversion.

If the request specifies a mode pack, the engine overrides default weights with the pack's custom table defined in open-sse/services/autoCombo/modePacks.ts.

Router Strategy Selection

After scoring, the system selects a provider using the configured RouterStrategy. The strategy is determined by the config.routerStrategy property (or legacy config.auto.routerStrategy), supporting implementations including:

• rules (default)
• cost
• latency
• sla-aware
• lkgp
• Custom strategies registered via registerStrategy

This selection process ensures the optimal candidate aligns with operational priorities, whether minimizing expense or maximizing response speed.

Configuring the Auto Routing Strategy

Model Prefix Variants

The open-sse/services/autoCombo/autoPrefix.ts module parses the auto/ suffix into a category and optional tier. Categories filter capabilities, while tiers apply specific weight profiles:

• Categories: coding, reasoning, vision, multimodal
• Tiers: fast, cheap, reliable, pro, free

For example, auto/coding:fast targets coding-capable models optimized for speed.

Router Strategy Configuration

Configure the routing behavior through the config.routerStrategy field when creating persisted auto combos via the API. Set this to cost to always select the cheapest healthy provider, latency for fastest response, or sla-aware for availability guarantees.

Per-Request Header Controls

Clients can override configuration using HTTP headers processed by open-sse/services/autoCombo/requestControls.ts:

• X-OmniRoute-Mode: Selects predefined weight profiles (fast, cheap, quality, offline, balanced, reliable)
• X-OmniRoute-Budget: Sets a hard USD cost ceiling (e.g., 0.05)
• X-OmniRoute-Budget-Fallback: Determines behavior when budget is exceeded—cheapest (default) selects the least expensive option, while strict returns HTTP 402

Implementation Examples

Zero-Config API Usage

Send requests without creating persistent combos:

// Basic auto routing for coding tasks
await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer <api-key>",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "auto/coding",
    messages: [{ role: "user", content: "Write a TypeScript function that sums an array." }],
  }),
});

Advanced Per-Request Controls

Override modes and budgets dynamically:

// Force fastest profile with strict $0.03 budget cap
await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer <api-key>",
    "Content-Type": "application/json",
    "X-OmniRoute-Mode": "fast",
    "X-OmniRoute-Budget": "0.03",
    "X-OmniRoute-Budget-Fallback": "strict",
  },
  body: JSON.stringify({
    model: "auto",
    messages: [{ role: "user", content: "Summarize the document." }],
  }),
});

Persistent Auto Combo Setup

Create reusable auto combos with custom strategies:

// Register a cost-optimized auto combo
await fetch("http://localhost:20128/api/combos", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    id: "my-auto",
    name: "Cost Optimized Auto",
    strategy: "auto",
    config: {
      routerStrategy: "cost",
      auto: {
        weights: { quota: 0.1, health: 0.4, costInv: 0.4, latencyInv: 0.1 },
      },
    },
  }),
});

Summary

• OmniRoute's auto routing strategy dynamically builds virtual combos at request time without database persistence.
• The system uses 9-factor or 14-factor scoring defined in open-sse/services/autoCombo/scoring.ts to rank candidates.
• Configuration occurs via model prefix variants (auto/category:tier), router strategy selection, and HTTP header overrides.
• Key source files include src/sse/handlers/chat.ts for detection and open-sse/services/autoCombo/virtualFactory.ts for candidate generation.

Frequently Asked Questions

What is the difference between auto routing and regular combos in OmniRoute?

Regular combos require manual provider configuration and persist in the database, while auto routing generates ephemeral virtual combinations on-the-fly. The auto strategy inspects all active connections in real-time through open-sse/services/autoCombo/virtualFactory.ts, whereas standard combos use predefined provider lists stored in the routing configuration.

How does OmniRoute handle budget constraints in auto routing?

The system checks the X-OmniRoute-Budget header against estimated costs before routing. If all candidates exceed the budget, the X-OmniRoute-Budget-Fallback header determines whether to select the cheapest available option or return an HTTP 402 status code, as implemented in open-sse/services/autoCombo/requestControls.ts.

Can I customize the scoring weights for auto routing?

Yes, you can override the DEFAULT_WEIGHTS by specifying custom weights in config.auto.weights when creating a persisted combo, or by using the X-OmniRoute-Mode header to select predefined mode packs from open-sse/services/autoCombo/modePacks.ts such as ship-fast, cost-saver, quality-first, or offline-friendly.

What router strategies are available for auto routing?

OmniRoute supports multiple RouterStrategy implementations including rules (default), cost, latency, sla-aware, and lkgp. You can also register custom strategies via the registerStrategy function and reference them in config.routerStrategy according to the strategy definitions in src/shared/constants/routingStrategies.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}