How OmniRoute's Auto-Combo Routing Works with Its 9-Factor Scoring System

OmniRoute dynamically selects the optimal LLM provider for each request by computing a weighted score across nine real-time telemetry factors, routing traffic to the highest-scoring candidate without requiring static configuration.

The auto-combo routing system in diegosouzapw/OmniRoute eliminates manual provider selection by analyzing live circuit-breaker states, quota availability, and latency metrics. When a request specifies a model prefixed with auto/, the system instantiates a virtual combo configuration and applies its multi-factor scoring algorithm to determine the best available provider. This implementation leverages the 9-factor scoring system defined in open-sse/services/autoCombo/scoring.ts to balance cost, performance, and reliability in real-time.

Auto-Combo Detection and Virtual Factory Creation

Request handling begins in src/sse/handlers/chat.ts, where the handler inspects the model name for the auto/ prefix. Upon detection, it delegates to virtualFactory.createVirtualAutoCombo() in open-sse/services/autoCombo/virtualFactory.ts, which constructs an in-memory AutoComboConfig without persisting to the database.

The factory queries all active provider connections via getProviderConnections({isActive:true}) and initializes a ProviderCandidate for each connection. As defined in open-sse/services/autoCombo/scoring.ts (lines 56-84), each candidate encapsulates:

• Circuit-breaker health state
• Remaining quota and rate-limit headroom
• Historical p95 latency and standard deviation
• Blended cost-per-million-tokens
• Account tier and task-fit metadata

The 9-Factor Scoring Algorithm

The scoring logic resides in scorePool() at open-sse/services/autoCombo/scoring.ts (line 15). This function computes a normalized score in the range [0,1] by calling calculateFactors (lines 78-112) to derive individual metrics, then passing these to calculateScore (lines 98-115) for weighted aggregation.

The default weights (DEFAULT_WEIGHTS, lines 41-54) assign non-zero values to nine primary factors that sum to 1.0:

• health (0.20): Circuit-breaker state where CLOSED = 1, HALF_OPEN = 0.5, and OPEN = 0
• quota (0.15): Percentage of remaining request quota or rate-limit capacity
• costInv (0.15): Inverse of blended cost-per-1M tokens (lower cost yields higher scores)
• latencyInv (0.12): Inverse of p95 latency in milliseconds
• taskFit (0.08): Alignment between model capabilities and inferred task type (coding, review, chat)
• stability (0.05): Inverse of latency standard deviation (consistency metric)
• tierPriority (0.05): Preference hierarchy (Ultra > Pro > Standard > Free)
• tierAffinity (0.05): Match between provider tier and manifest-hinted minimum tier
• specificityMatch (0.05): Correlation between provider tier and request specificity score

Two additional factors—resetWindowAffinity and connectionDensity—maintain default weights of 0.0 but can be activated via custom mode packs.

Selection Flow and Mode Pack Overrides

The routing pipeline executes four discrete phases:

1. Candidate Pool Construction: buildAutoCandidates() in open-sse/services/combo.ts (lines 82-94) aggregates pricing, telemetry, and health data for every active connection.
2. Weight Modification: Mode packs defined in open-sse/services/autoCombo/modePacks.ts (e.g., ship-fast, cost-saver, quality-first) override default weights to bias selection. For example, ship-fast increases latencyInv weight while reducing costInv emphasis.
3. Score Computation: scorePool() calculates ScoringFactors for each candidate and applies the weighted sum formula.
4. Top-Candidate Selection: handleComboChat (lines 1020-1060) routes the request to the highest-scoring provider in the sorted pool.

Optional router strategies (routerStrategy in combo config) can re-rank candidates before final selection. The default rules strategy returns the top-scoring candidate, while alternatives like cost, latency, or sla-aware apply additional filtering layers.

Self-Healing and Bandit Exploration

The system implements resilience mechanisms through open-sse/services/autoCombo/selfHealing.ts. Candidates scoring below 0.2 enter a 5-minute backoff period, temporarily removing them from the pool to prevent cascading failures.

A bandit-exploration mechanism (default 5% of requests) injects random candidate selection to discover potentially optimal providers that might otherwise rank lower due to historical data sparsity. This exploration disables automatically during incident mode.

Practical Implementation Examples

Zero-Config Auto-Routing with curl

curl -sS http://localhost:20128/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto/coding",
        "messages": [{"role": "user", "content": "Write a Python function to compute the Fibonacci sequence"}]
      }'

The auto/coding variant automatically applies the quality-first mode pack, prioritizing task-fit and stability over raw latency.

Selecting Mode Packs via Headers

curl -sS http://localhost:20128/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -H "X-OmniRoute-Mode: fast" \
  -H "X-OmniRoute-Budget: 0.03" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "What is the weather in Paris?"}]
      }'

The X-OmniRoute-Mode: fast header activates the ship-fast weight pack, while X-OmniRoute-Budget: 0.03 filters out candidates exceeding $0.03 per request before scoring begins.

Persisted Auto-Combo Configuration

POST /api/combos
{
  "id": "my-auto",
  "name": "My Auto Combo",
  "strategy": "auto",
  "config": {
    "auto": {
      "candidatePool": ["openai", "anthropic", "google"],
      "weights": {
        "quota": 0.15,
        "health": 0.3,
        "costInv": 0.05,
        "latencyInv": 0.35,
        "taskFit": 0.1,
        "stability": 0,
        "tierPriority": 0.05
      }
    }
  }
}

Clients can now reference "model": "my-auto" to utilize this custom weight profile while maintaining the same underlying scoring pipeline.

Custom Router Strategy in TypeScript

import { registerStrategy, type RouterStrategy } from "@omniroute/open-sse/services/autoCombo/routerStrategy";

class Sub10msStrategy implements RouterStrategy {
  readonly name = "sub10ms";
  readonly description = "Prefers providers with <10ms latency and <1% error rate";

  select(pool, _ctx) {
    const healthy = pool.filter(c => c.circuitBreakerState !== "OPEN");
    const filtered = healthy.filter(c => c.p95LatencyMs < 10 && c.errorRate < 0.01);
    return filtered.length ? filtered[0] : healthy[0];
  }
}

registerStrategy("sub10ms", new Sub10msStrategy());

Activate this strategy by setting "config.routerStrategy": "sub10ms" in your combo configuration.

Summary

• Auto-combo routing triggers when model names use the auto/ prefix, creating ephemeral configurations via virtualFactory.createVirtualAutoCombo()
• The 9-factor scoring system evaluates health (0.20), quota (0.15), cost (0.15), latency (0.12), task-fit (0.08), stability (0.05), and three tier-related factors (0.05 each) to generate scores between 0 and 1
• Mode packs in open-sse/services/autoCombo/modePacks.ts allow runtime weight adjustments via headers like X-OmniRoute-Mode: fast
• Candidates scoring below 0.2 enter a 5-minute healing cooldown, while 5% of requests use bandit exploration to optimize provider discovery
• Custom router strategies enable advanced filtering logic beyond the standard weighted scoring algorithm

Frequently Asked Questions

How are the 9 factors weighted in OmniRoute's scoring system?

The default weights are defined in open-sse/services/autoCombo/scoring.ts (lines 41-54) as constants summing to 1.0. Health receives the highest weight at 0.20, followed by quota and cost (0.15 each), latency (0.12), and task-fit (0.08). The remaining four factors—stability, tier priority, tier affinity, and specificity match—each carry 0.05 weight. These values can be overridden via mode packs or custom combo configurations.

Can I customize which factors matter most for my specific workload?

Yes. You can supply custom weights through the weights field in a persisted combo configuration, or use the X-OmniRoute-Mode header to select pre-defined mode packs. For example, setting "mode": "cost-saver" increases the costInv weight while reducing latencyInv emphasis, whereas "mode": "ship-fast" inverts these priorities. You can also implement a custom RouterStrategy to bypass the weighted scoring entirely and use deterministic logic.

What happens when all available providers are unhealthy?

If all candidates report circuit-breaker states of OPEN or HALF_OPEN (health scores of 0 or 0.5), the system still routes to the highest-scoring candidate among them, typically prioritizing those in HALF_OPEN state. Additionally, the self-healing mechanism in autoCombo/selfHealing.ts temporarily excludes candidates scoring below 0.2 for 5 minutes, allowing degraded providers to recover before re-entering the pool.

How does auto-combo routing differ from static combo configurations?

Static combos require manual database entries specifying fixed provider lists and fallback chains. Auto-combo configurations are virtual—they exist only in memory during request processing and are built dynamically from all active provider connections. This eliminates configuration drift and allows the system to automatically incorporate new providers or remove failed ones without updating database records.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}