How OmniRoute's Auto-Combo Scoring Works with Its 12-Factor Evaluation System

OmniRoute's auto-combo engine selects the best provider-model pair by calculating a weighted score from 12 normalized factors, including two "I2" inverse metrics (costInv and latencyInv) that reward cheaper and faster providers.

The auto-combo scoring system in OmniRoute is the decision-making core that ranks AI provider candidates in real-time. This article breaks down how the 12-factor evaluation works, with special focus on the I2 inverse factors that directly impact cost and latency optimization.

The Auto-Combo Scoring Pipeline

When a request hits the auto-combo router, viable ProviderCandidate objects flow through a three-stage pipeline defined in open-sse/services/autoCombo/scoring.ts:

1. scorePool() (lines 41-48) — Entry point that iterates candidates and orchestrates scoring
2. calculateFactors() (lines 20-38) — Normalizes raw telemetry to [0..1] range
3. calculateScore() (lines 22-41) — Multiplies factors by weights and sums to final score

Each stage feeds into the next, with the final score clamped to [0..1] to prevent malformed data from corrupting rankings.

// Scoring flow as implemented in the source
const scored = scorePool(pool, "chat", DEFAULT_WEIGHTS, getTaskFitness);
// Returns: Array of { candidate, score, factors }

Understanding the 12 Scoring Factors

OmniRoute evaluates candidates across 12 weighted factors. While the complete list includes quota remaining, circuit breaker state, task fitness, and provider health, this analysis focuses on the I2 pair — the inverse-cost and inverse-latency metrics that optimize for economic and performance efficiency.

Factor Category	Examples	Default Weight
Capacity factors	quotaRemaining, circuitBreakerState	Varies
Quality factors	taskFit, healthScore	Varies
I2 factors	costInv, latencyInv	0.15, 0.12
Stability factors	errorRate, successRate	Varies

Weights are configurable per auto-combo configuration and automatically normalized to sum to ≈1 via normalizeScoringWeights().

The I2 Factors: Inverse-Cost and Inverse-Latency

The I2 factors derive their name from being the second pair of "inverse" metrics in the research paper that inspired OmniRoute's design. These factors invert cost and latency so that lower values produce higher scores.

costInv: Cheaper Providers Score Higher

// Line 27 of calculateFactors
const costInv = clamp01(1 - candidate.costPer1MTokens / maxCost);

• maxCost = highest cost in the current candidate pool
• A provider at 50% of max cost earns costInv = 0.5
• The cheapest provider earns costInv ≈ 1.0

latencyInv: Faster Providers Score Higher

// Line 28 of calculateFactors
const latencyInv = clamp01(1 - candidate.p95LatencyMs / maxLatency);

• maxLatency = highest P95 latency in the pool
• Uses P95 latency to avoid outlier sensitivity
• Fastest provider earns latencyInv ≈ 1.0

Both values pass through clamp01() (defined in open-sse/utils/number.ts, lines 24-27) as a safety guard against division-by-zero or negative telemetry.

Default Weights and Customization

The DEFAULT_WEIGHTS object (lines 43-48 of scoring.ts) assigns:

Factor	Default Weight	Purpose
costInv	0.15	Moderate preference for cost efficiency
latencyInv	0.12	Slight preference for speed

These defaults balance cost, latency, and quality without over-prioritizing any single dimension.

Overriding Weights Per Combo

import { normalizeScoringWeights, scorePool } from "@omniroute/open-sse/services/autoCombo/scoring";

// Prioritize latency for real-time applications
const realtimeWeights = {
  ...DEFAULT_WEIGHTS,
  costInv: 0.08,      // reduce cost priority
  latencyInv: 0.25,   // boost latency priority
};

const normalized = normalizeScoringWeights(realtimeWeights);
const scored = scorePool(pool, "chat", normalized, getTaskFitness);

The normalizeScoringWeights() function rescales any custom set so weights sum to 1.0, preserving proportional relationships between factors.

Impact on Routing Decisions

The I2 factors directly shape which provider-model pair wins selection:

• Cheap + Fast provider: costInv ≈ 1, latencyInv ≈ 1 → high composite score even with average other factors
• Expensive or Slow provider: costInv ≈ 0 or latencyInv ≈ 0 → score dragged down despite strong quota or health metrics
• Edge case protection: clamp01() ensures no single malformed metric (negative cost, zero latency) can crash the scoring engine

After scoring, pipelineRouter.ts sorts candidates descending by score and executes fallback logic if the top choice is unavailable.

Complete Working Example

import { 
  scorePool, 
  DEFAULT_WEIGHTS, 
  normalizeScoringWeights 
} from "@omniroute/open-sse/services/autoCombo/scoring";
import { getTaskFitness } from "@omniroute/open-sse/services/autoCombo/taskFitness";

// Build candidate pool from telemetry
const pool: ProviderCandidate[] = [
  {
    provider: "openai",
    model: "gpt-4o",
    costPer1MTokens: 0.028,
    p95LatencyMs: 180,
    quotaRemaining: 85,
    circuitBreakerState: "CLOSED",
    // ... additional fields
  },
  {
    provider: "anthropic",
    model: "claude-3-5-sonnet",
    costPer1MTokens: 0.018,
    p95LatencyMs: 220,
    quotaRemaining: 92,
    circuitBreakerState: "CLOSED",
  }
];

// Score with defaults
const scored = scorePool(pool, "chat", DEFAULT_WEIGHTS, getTaskFitness);

// Inspect I2 contributions for top candidate
const best = scored[0];
console.log(`${best.candidate.provider}: score=${best.score.toFixed(3)}`);
console.log(`  costInv=${best.factors.costInv.toFixed(2)}, latencyInv=${best.factors.latencyInv.toFixed(2)}`);

// Output might show:
// anthropic: score=0.847
//   costInv=1.00, latencyInv=0.18

Key Source Files

File	Responsibility	Lines of Interest
open-sse/services/autoCombo/scoring.ts	Core scoring logic	20-48 (factors, weights, aggregation)
open-sse/services/autoCombo/engine.ts	Candidate pool assembly	Full orchestration
open-sse/services/autoCombo/pipelineRouter.ts	Final selection & fallback	Sorting and routing
open-sse/services/autoCombo/taskFitness.ts	Task-fit factor calculation	Quality scoring
open-sse/utils/number.ts	clamp01 safety guard	24-27

Summary

• OmniRoute's auto-combo scoring evaluates 12 normalized factors to rank provider-model candidates
• The I2 factors (costInv, latencyInv) invert cost and latency so cheaper/faster providers score higher
• Default weights (0.15 for cost, 0.12 for latency) are customizable per combo via normalizeScoringWeights()
• All factor values pass through clamp01() for defensive programming against malformed telemetry
• The scoring pipeline lives in scoring.ts and is orchestrated by engine.ts and pipelineRouter.ts

Frequently Asked Questions

What does "I2" stand for in OmniRoute's scoring system?

I2 refers to the second pair of "inverse" metrics in the research paper that inspired OmniRoute's design: inverse-cost (costInv) and inverse-latency (latencyInv). These factors transform raw cost and latency values so that lower input values produce higher scores, aligning the mathematics with the business goal of preferring cheaper and faster providers.

How are costInv and latencyInv calculated from raw telemetry?

Both factors use pool-relative normalization. For costInv, the engine computes 1 - (candidate.cost / maxCostInPool) and clamps to [0,1]. For latencyInv, it uses 1 - (candidate.p95LatencyMs / maxLatencyInPool) with the same clamping. This ensures every candidate is scored relative to its current competition, not against absolute global thresholds.

Can I disable or zero out the I2 factors if cost and latency don't matter?

Yes. Pass a custom weights object to scorePool() with costInv: 0 and latencyInv: 0, then run normalizeScoringWeights() to rebalance remaining factors. The normalization redistributes the freed weight proportionally across other factors like taskFit and healthScore.

Why does OmniRoute use P95 latency instead of average latency?

P95 latency (the 95th percentile) filters out tail-end outliers that would skew average-based scoring. This protects against providers with generally fast responses but occasional extreme slowdowns — a critical reliability consideration for production AI routing where consistent performance matters more than best-case performance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}