How OmniRoute's Auto-Combo Routing Uses the 12‑Factor Scoring System

OmniRoute's auto-combo routing dynamically selects the best provider-model pair by scoring candidates against 12 weighted runtime factors—including latency, reliability, quota status, cost, and cache affinity—to maximize performance while respecting constraints.

The auto-combo feature in diegosouzapw/OmniRoute eliminates manual model selection. When a request omits a specific model (or requests "auto"), the engine constructs a virtual combo and applies a 12‑factor scoring system to rank candidates. This article explains how the scoring pipeline works, where each factor originates, and how you can influence selection through mode packs.

---

What Is Auto‑Combo Routing?

Auto-combo routing is OmniRoute's intelligent fallback mechanism. Instead of failing when a preferred model is unavailable, the system builds a candidate pool of viable provider-model combinations and selects the highest-scoring option. This happens in two phases:

• Pool construction – assembles all eligible providers based on connection status, authentication requirements, and availability flags.
• Candidate scoring – applies the 12‑factor algorithm to compute a normalized score for each candidate.

The routing decision occurs at request time with negligible latency overhead.

---

Building the Candidate Pool

The virtual combo factory in open-sse/services/autoCombo/virtualFactory.ts generates candidate objects on-the-fly. These virtual combos bypass database lookups but retain full scoring capabilities. The factory filters providers according to:

• Connection health (established sockets or valid no-auth configuration)
• Temporary ban status (circuit breaker state)
• Minimum capability requirements for the requested task

Providers passing these filters enter the scoring pipeline as candidate objects with initialized metadata structures.

---

The 12‑Factor Scoring Pipeline

The core scoring logic resides in open-sse/services/combo/autoStrategy.ts, with factor definitions and weight configurations in src/lib/combos/intelligentRouting.ts. Each candidate receives a score between 0 and 1 derived from twelve weighted factors.

Primary Factors (Latency & Reliability)

Factor	Source File	Description
P50 Latency	src/lib/usage/usageHistory.ts	50th percentile response time from historical telemetry
P99 Latency	src/lib/usage/usageHistory.ts	99th percentile for tail latency sensitivity
Success Rate	src/lib/usage/usageHistory.ts	Ratio of successful responses to total requests
Error Rate Decay	src/lib/usage/usageHistory.ts	Time-weighted error frequency with exponential decay

These four factors constitute the reliability core of the scoring system, directly impacting user-perceived quality.

Resource & Constraint Factors

Factor	Source File	Description
Quota Soft Penalty	open-sse/services/combo/autoStrategy.ts	Progressive penalty as provider approaches rate limits
Hard Quota Exhaustion	open-sse/services/combo/autoStrategy.ts	Binary exclusion when quota fully consumed
Concurrent Request Load	open-sse/services/combo/autoStrategy.ts	Penalty based on active request saturation
Cost Per 1M Tokens	src/lib/usage/usageHistory.ts	Normalized pricing telemetry for budget-aware routing

The quota soft penalty implementation applies non-linear scaling:

// From open-sse/services/combo/autoStrategy.ts
function computeQuotaSoftPenalty(
  remainingQuota: number,
  totalQuota: number,
  config: QuotaPenaltyConfig
): number {
  const utilization = 1 - (remainingQuota / totalQuota);
  // Exponential penalty curve: 0 → 0.2 → 0.5 → 0.9 as utilization approaches 1
  return Math.pow(utilization, config.exponent) * config.maxPenalty;
}

Optimization Factors

Factor	Source File	Description
Cache Affinity	open-sse/services/combo/promptCacheAffinity.ts	Probability of cache hit for prompt similarity
Token Throughput	src/lib/usage/usageHistory.ts	Observed tokens-per-second processing rate
Provider Preference Weight	src/lib/combos/intelligentRouting.ts	Administrator-configured base weighting
Model Capability Match	src/lib/combos/intelligentRouting.ts	Feature alignment with request requirements

Score Aggregation

The final composite score combines all twelve factors through weighted summation:

// Simplified from intelligentRouting.ts
interface ScoringWeights {
  p50Latency: number;
  p99Latency: number;
  successRate: number;
  errorRateDecay: number;
  quotaSoftPenalty: number;
  hardQuotaExhaustion: number;
  concurrentLoad: number;
  costPer1MTokens: number;
  cacheAffinity: number;
  tokenThroughput: number;
  providerPreference: number;
  capabilityMatch: number;
}

function calculateCompositeScore(
  candidate: Candidate,
  factors: FactorValues,
  weights: ScoringWeights
): number {
  const weightedSum = 
    factors.p50Latency * weights.p50Latency +
    factors.p99Latency * weights.p99Latency +
    factors.successRate * weights.successRate +
    factors.errorRateDecay * weights.errorRateDecay +
    (1 - factors.quotaSoftPenalty) * weights.quotaSoftPenalty +
    (factors.hardQuotaAvailable ? 1 : 0) * weights.hardQuotaExhaustion +
    (1 - factors.concurrentLoad) * weights.concurrentLoad +
    (1 - normalizedCost(factors.costPer1MTokens)) * weights.costPer1MTokens +
    factors.cacheAffinity * weights.cacheAffinity +
    normalizedThroughput(factors.tokenThroughput) * weights.tokenThroughput +
    factors.providerPreference * weights.providerPreference +
    factors.capabilityMatch * weights.capabilityMatch;
  
  return Math.min(Math.max(weightedSum, 0), 1);
}

Each factor is normalized to [0, 1] before weighting. Higher scores indicate better candidates. The engine selects the candidate with maximum composite score, breaking ties via deterministic hash of candidate ID.

---

Mode Packs: Controlling Scoring Priorities

The 12‑factor system exposes tunable weights through mode packs—named configuration presets that shift scoring emphasis. Mode packs are resolved from:

• The X-OmniRoute-Mode HTTP header
• Query parameter ?mode=
• Default configuration in src/lib/combos/intelligentRouting.ts

Built-in Mode Packs

Mode Pack	Emphasis	Typical Weights
default	Balanced	Equal weight on latency, reliability, cost
latency-priority	Minimize response time	p50Latency: 0.25, p99Latency: 0.20, successRate: 0.15
cost-optimized	Minimize spend	costPer1MTokens: 0.30, tokenThroughput: 0.15
reliability-first	Maximize uptime	successRate: 0.30, errorRateDecay: 0.25
cache-maximize	Exploit prompt caching	cacheAffinity: 0.35, p50Latency: 0.10

Custom mode packs can be defined in configuration files and referenced by name.

---

Fallback Handling and Degradation

When the top-scoring candidate fails during execution, the engine enters fallback mode:

1. Increase quotaSoftPenalty weight for the failed provider (temporary negative memorization)
2. Re-run scoring with updated weights
3. Select next-best candidate from remaining pool

If all candidates exhaust, OmniRoute returns HTTP 503 with diagnostic information from src/lib/usage/comboScoringInspector.ts.

---

Observability and Debugging

The scoring inspector in src/lib/usage/comboScoringInspector.ts generates detailed factor breakdowns for every routing decision when debug mode is enabled.

// Request with debug flag
const response = await fetch("/api/v1/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-OmniRoute-Mode": "latency-priority",
    "X-OmniRoute-Debug": "true"
  },
  body: JSON.stringify({ prompt: "Analyze this dataset." })
});

// Response includes X-OmniRoute-Scoring-Trace header

The trace contains per-candidate factor values and weighted contributions, enabling performance analysis and mode pack tuning.

---

Practical Examples

Basic Auto-Combo Request

// No model specified → triggers 12-factor scoring
await fetch("https://api.omniroute.io/v1/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    prompt: "Summarize the key points of quantum computing.",
    max_tokens: 500
  })
});

Cost-Conscious Routing

// Prefer cheaper providers if latency remains acceptable
await fetch("https://api.omniroute.io/v1/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-OmniRoute-Mode": "cost-optimized"
  },
  body: JSON.stringify({
    prompt: "Generate ten product descriptions.",
    max_tokens: 2000
  })
});

Retrieving Scoring Diagnostics

# Admin endpoint for candidate inspection

curl "https://api.omniroute.io/v1/admin/auto-combo/candidates?channel=production&debug=true" \
  -H "Authorization: Bearer $ADMIN_TOKEN"

Response includes complete 12-factor breakdown for each candidate in the pool.

---

Key Source Files

File	Purpose	Link
open-sse/services/autoCombo/virtualFactory.ts	Candidate pool construction	View
open-sse/services/combo/autoStrategy.ts	Core scoring algorithm, quota penalties	View
src/lib/combos/intelligentRouting.ts	Mode pack definitions, weight vectors	View
src/lib/usage/usageHistory.ts	Latency, reliability, cost telemetry	View
open-sse/services/combo/promptCacheAffinity.ts	Cache hit probability scoring	View
src/lib/usage/comboScoringInspector.ts	Diagnostic trace generation	View

---

Summary

• Auto-combo routing eliminates manual model selection by dynamically building candidate pools and applying algorithmic scoring.
• The 12‑factor scoring system evaluates candidates across latency, reliability, quota status, cost, cache affinity, and operational metrics.
• Mode packs provide tunable weight presets (X-OmniRoute-Mode) to shift routing behavior toward latency, cost, reliability, or cache optimization.
• Fallback logic gracefully degrades to lower-scored candidates when preferred options fail.
• Full observability through comboScoringInspector.ts enables performance tuning and debugging of routing decisions.

---

Frequently Asked Questions

What happens when multiple candidates have identical 12‑factor scores?

Ties are broken using a deterministic hash of the candidate identifier combined with the request timestamp. This ensures consistent routing for repeated identical requests while distributing load across equivalent candidates over time.

Can I exclude specific factors from the scoring calculation?

Not individually at the request level. However, you can create a custom mode pack with zero weights for undesired factors, effectively removing their influence. Mode packs are defined in configuration and referenced via X-OmniRoute-Mode.

How quickly does the quota soft penalty adjust to changing limits?

The penalty recalculates on every request using real-time quota telemetry from usageHistory.ts. The exponential curve in autoStrategy.ts ensures responsive adjustment as utilization exceeds 70%, with full penalty approaching at 95%+ consumption.

Does the 12‑factor system work with provider-specific authentication requirements?

Yes. Authentication validation occurs before pool construction in virtualFactory.ts. Only providers with valid credentials (or allow-listed no-auth endpoints) enter the candidate pool, so scoring operates exclusively on viable options.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}