How the OmniRoute Auto-Combo Engine Scores and Selects the Best Model Using 14 Factors

The OmniRoute Auto-Combo engine scores providers using the I4 factors—Quota, Health, Cost⁻¹, and Latency⁻¹—then applies 10 additional secondary factors to compute a weighted composite score, ranks candidates by score, and selects the top performer.

OmniRoute's Auto-Combo engine implements intelligent combo-routing that dynamically evaluates provider-model pairs at request time. The engine's scoring system centers on four core "I4" influence factors—Quota, Health, Cost Inverse, and Latency Inverse—supplemented by up to 10 additional optional factors for specialized routing scenarios. This article breaks down the complete scoring pipeline as implemented in the diegosouzapw/OmniRoute repository.

Understanding the I4 Core Factors

The I4 factors form the foundation of OmniRoute's scoring algorithm. Each factor normalizes a critical provider characteristic to a 0-1 scale:

Factor	Calculation	Data Source
Quota	quotaRemaining / quotaTotal	ProviderCandidate.quotaRemaining
Health	Circuit-breaker health rating	ProviderCandidate.circuitBreakerState
Cost Inverse	1 / costPer1MTokens	ProviderCandidate.costPer1MTokens
Latency Inverse	1 / p95LatencyMs	ProviderCandidate.p95LatencyMs

Higher values indicate better provider fitness. Cheaper providers and lower-latency endpoints receive elevated scores through their inverse calculations.

The Full 14-Factor Scoring System

Beyond the I4 core, OmniRoute's scoring pipeline supports 10 additional optional factors defined in the ScoringFactors interface:

• taskFit — alignment between model capabilities and request type
• stability — historical uptime consistency
• tierScore — provider tier classification
• contextWindowFit — appropriateness of model's context window
• throughput — tokens-per-second capacity
• errorRate — recent failure percentage
• coldStartLatency — time to first token
• geographicProximity — network distance to request origin
• complianceScore — regulatory alignment for sensitive data
• customFactor — user-defined extension point

These factors enable fine-grained routing for specialized workloads while maintaining lightweight I4-only scoring for standard requests.

Weight Normalization and Default Configuration

UI-Provided Weights

Custom weights from the UI undergo normalization via normalizeScoringWeights to ensure valid probability distributions. The function redistributes any negative weights and rescales so all weights sum to 1.0.

Default I4 Weights

When no custom weights exist, the engine uses DEFAULT_WEIGHTS defined in open-sse/services/autoCombo/scoring.ts (lines 45-60):

export const DEFAULT_WEIGHTS: ScoringWeights = {
  quota: 0.35,      // 35% — prevent quota exhaustion
  health: 0.30,     // 30% — prioritize reliable providers
  costInv: 0.20,    // 20% — moderate cost sensitivity
  latencyInv: 0.15, // 15% — balance speed vs. cost
};

The default distribution emphasizes quota preservation and health stability over pure cost optimization.

Score Calculation Pipeline

Step 1: Factor Collection

For each ProviderCandidate, the engine assembles a ScoringFactors object with available metrics.

Step 2: Weighted Sum Computation

The calculateScore function (located at open-sse/services/autoCombo/scoring.ts, lines 14-18) computes the composite:

export function calculateScore(
  factors: ScoringFactors, 
  weights: ScoringWeights
): number {
  return clamp01(
    weights.quota * factors.quota +
    weights.health * factors.health +
    weights.costInv * factors.costInv +
    weights.latencyInv * factors.latencyInv +
    (weights.taskFit || 0) * (factors.taskFit || 0) +
    (weights.stability || 0) * (factors.stability || 0) +
    // ...additional factors...
  );
}

Step 3: Safe Value Clamping

The clamp01 utility guarantees output in [0, 1] and handles edge cases:

• Non-finite results (NaN, Infinity) map to 0
• Values below 0 clamp to 0
• Values above 1 clamp to 1

Step 4: Ranking and Selection

The auto-strategy implementation in open-sse/services/combo/autoStrategy.ts (lines 390-405) orchestrates final selection:

// Build scored candidates
const scoredCandidates = candidates.map(c => ({
  provider: c.provider,
  model: c.model,
  score: calculateScore(c.factors, normalizedWeights),
  factors: c.factors
}));

// Descending sort by composite score
scoredCandidates.sort((a, b) => b.score - a.score);

const bestModel = scoredCandidates[0];

Tie-breaking falls back to original candidate order when scores match.

Practical Implementation Example

Basic I4 Scoring

import {
  calculateScore,
  DEFAULT_WEIGHTS,
  normalizeScoringWeights,
  type ScoringFactors,
} from '@/open-sse/services/autoCombo/scoring';

const candidateFactors: ScoringFactors = {
  quota: 0.78,          // 78% quota remaining
  health: 0.92,         // Strong health rating
  costInv: 1 / 0.0012,  // ~833 inverse cost
  latencyInv: 1 / 120,  // ~0.0083 inverse latency (ms)
};

const score = calculateScore(candidateFactors, DEFAULT_WEIGHTS);
console.log(`I4 composite score: ${score.toFixed(4)}`); // e.g., 0.8473

Full 14-Factor Routing

import { resolveCandidates } from '@/open-sse/services/combo/autoStrategy';

async function selectOptimalProvider(request) {
  // Resolve all candidate providers
  const candidates = await resolveCandidates(request);
  
  // Apply custom weights emphasizing cost and latency
  const customWeights = normalizeScoringWeights({
    quota: 0.25,
    health: 0.25,
    costInv: 0.30,
    latencyInv: 0.15,
    taskFit: 0.05,
  });
  
  // Score and rank
  const ranked = candidates
    .map(c => ({
      ...c,
      score: calculateScore(c.factors, customWeights)
    }))
    .sort((a, b) => b.score - a.score);
  
  return ranked[0]; // Optimal provider-model pair
}

Key Source Files

File	Purpose
open-sse/services/autoCombo/scoring.ts	Core scoring logic: calculateScore, DEFAULT_WEIGHTS, normalizeScoringWeights, ScoringFactors interface
open-sse/services/combo/autoStrategy.ts	Candidate resolution, factor assembly, ranking, and selection
tests/unit/auto-combo-engine.test.ts	Unit tests validating I4 factor calculations and edge cases
docs/routing/AUTO-COMBO.md	Architectural documentation for the Auto-Combo system

All references correspond to release v3.8.50 of the OmniRoute repository.

Summary

• I4 factors (Quota, Health, Cost⁻¹, Latency⁻¹) form the mandatory core of OmniRoute's scoring system
• 10 optional factors extend routing intelligence for specialized workloads without mandatory overhead
• calculateScore computes weighted sums with safe clamp01 normalization to [0, 1]
• DEFAULT_WEIGHTS prioritizes quota preservation (35%) and health (30%) over cost (20%) and latency (15%)
• normalizeScoringWeights ensures valid probability distributions from UI or API inputs
• Descending sort by composite score with fallback tie-breaking yields the optimal provider-model selection

Frequently Asked Questions

What does "I4" stand for in OmniRoute's scoring system?

I4 refers to the four influence factors: Quota, Health, Cost Inverse, and Latency Inverse. The name originates from internal design documentation grouping these as the most influential dimensions for provider selection. These four factors are inexpensive to compute at runtime and capture the essential tradeoffs between resource availability, reliability, price, and speed.

How does OmniRoute handle missing or incomplete factor data?

Missing optional factors default to 0 in the weighted sum, effectively excluding them from scoring. The calculateScore function uses safe multiplication (weights.factor || 0) * (factors.factor || 0) for all secondary factors. Core I4 factors are required; if any are undefined, the candidate receives a score of 0 via clamp01's NaN handling, preventing unstable providers from ranking highly.

Can I customize which factors matter most for my workload?

Yes, through normalizeScoringWeights you can supply custom weights for any combination of the 14 factors. The normalization function redistributes values to ensure a valid probability distribution. For example, batch processing jobs might increase costInv weight to 0.40 while real-time applications might prioritize latencyInv at 0.35. Weights not specified default to 0.

Why use inverse values for cost and latency instead of direct values?

Inverse transformation ensures higher scores represent better performance while maintaining proportional sensitivity. Direct latency values would require complex decreasing functions; 1/latency naturally maps faster providers to higher scores with appropriate scaling. This approach also ensures that when latency approaches zero (ideal), the score contribution grows appropriately without arbitrary caps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}