How OmniRoute's Auto-Combo Engine Scores Providers Using Its 14-Factor Scoring System

The OmniRoute Auto-Combo Engine ranks AI providers using a weighted sum algorithm that evaluates four primary signals—quota availability, health status, token cost, and response latency—applying default weights of 0.30, 0.30, 0.20, and 0.20 to calculate a final composite score.

The OmniRoute Auto-Combo Engine (from the diegosouzapw/OmniRoute repository) intelligently routes LLM requests by dynamically selecting optimal provider-model combinations. While the architecture supports a comprehensive 14-factor evaluation framework, the production implementation uses a lightweight I4-factor scoring function that focuses on four critical operational metrics to balance computational efficiency with routing precision.

Understanding the I4-Factor Scoring Variant

The I4-factor system represents the operational subset of OmniRoute's full 14-factor scoring engine. This lightweight variant activates by default when clients specify model: "auto" without requesting advanced routing heuristics. The engine evaluates each candidate provider through four normalized dimensions before applying configurable weights to determine the winning connection.

The Four Core Scoring Factors

Each provider candidate is assessed across these quantitative signals:

• Quota (Weight: 0.30) — Measures remaining usable capacity via candidate.quotaAvailable, derived from the provider-level quota cache. Higher available quota increases the score.

• Health (Weight: 0.30) — Reflects recent success rates through candidate.healthScore, a normalized 0-1 value maintained by the resilience layer in src/lib/resilience. This incorporates circuit-breaker status and connection cooldowns.

• Cost (Weight: 0.20) — Evaluates monetary efficiency using candidate.costUsdPerK, representing the price per 1,000 tokens retrieved from the provider model catalog.

• Latency (Weight: 0.20) — Tracks observed round-trip performance via candidate.latencyMs, a moving average maintained by the request-monitoring subsystem that includes any active back-off delays.

Technical Implementation in scoring.ts

The scoring logic resides in open-sse/services/autoCombo/scoring.ts, which exports the calculateScore() function and default weight configurations.

Default Weight Configuration

The system defines baseline weights through the DEFAULT_WEIGHTS constant:

// open-sse/services/autoCombo/scoring.ts
export const DEFAULT_WEIGHTS: ScoringWeights = {
  quota: 0.30,
  health: 0.30,
  cost: 0.20,
  latency: 0.20,
  // Additional factors zeroed for I4 mode
};

Score Calculation Algorithm

The calculateScore() function computes the weighted sum after normalizing raw metrics to the [0, 1] range:

// open-sse/services/autoCombo/scoring.ts
export function calculateScore(
  candidate: ProviderCandidate,
  weights: ScoringWeights,
): number {
  // Normalize factors to comparable scales
  const fQuota   = normalizeQuota(candidate.quotaAvailable);
  const fHealth  = candidate.healthScore;               // pre-normalized
  const fCost    = normalizeCost(candidate.costUsdPerK);
  const fLatency = normalizeLatency(candidate.latencyMs);

  // Calculate weighted composite score
  return (
    fQuota   * weights.quota   +
    fHealth  * weights.health  +
    fCost    * weights.cost    +
    fLatency * weights.latency
  );
}

The normalization functions (normalizeQuota(), normalizeCost(), normalizeLatency()) ensure that each factor contributes proportionally regardless of its original unit scale (milliseconds vs. dollars vs. token counts).

End-to-End Provider Selection Workflow

The Auto-Combo Engine executes a five-stage pipeline to select the optimal provider:

1. Candidate Generation — The virtualFactory.ts module collects all provider-model combinations matching the request's model pattern, including auto-combo variants.

2. Filtering — Candidates blocked by circuit-breakers, exhausted quota limits, or explicit disable flags are removed from consideration.

3. Scoring — The scorePool() function (re-exported from scoring.ts) invokes calculateScore() for each surviving candidate using the active weight vector.

4. Sorting — Candidates are ordered by descending composite score; the top-ranked entry receives the request.

5. Fallback — If the primary selection fails, the engine automatically retries with the next candidate in the ranked list, preserving the I4-factor ordering.

Customizing Scoring Weights

Developers can override the default I4 weights to emphasize specific operational priorities. For example, to prioritize cost reduction over latency:

import { scorePool } from '@/open-sse/services/autoCombo/scoring';
import { fetchCandidates } from '@/open-sse/services/autoCombo/virtualFactory';

const candidates = await fetchCandidates(request);

// Custom I4 weights favoring cost efficiency
const costOptimizedWeights = {
  quota: 0.25,
  health: 0.25,
  cost: 0.40,    // Increased cost influence
  latency: 0.10, // Reduced latency priority
};

const scoredProviders = scorePool(candidates, costOptimizedWeights);
const bestProvider = scoredProviders[0];

Integration with Router Strategy

The open-sse/services/autoCombo/routerStrategy.ts file orchestrates the scoring invocation. It determines whether to apply the full 14-factor analysis or the lightweight I4 variant based on the request configuration. For standard auto-routing scenarios, it delegates to the I4-factor implementation to minimize per-request overhead while maintaining intelligent provider selection.

Summary

• OmniRoute uses a lightweight I4-factor scoring system (Quota, Health, Cost, Latency) as the operational implementation of its broader 14-factor architecture.
• Default weights prioritize quota availability and health (30% each) over cost and latency (20% each).
• The calculateScore() function in scoring.ts computes a weighted sum of normalized factor values.
• Raw metrics are normalized via dedicated functions before weighting to ensure fair comparison across different scales.
• The system supports custom weight configurations to optimize for cost-sensitive or latency-critical workloads.
• Failed requests trigger automatic fallback to the next highest-ranked candidate in the scored pool.

Frequently Asked Questions

What is the difference between I4-factor and 14-factor scoring in OmniRoute?

The I4-factor system is a lightweight operational mode that evaluates four core signals (quota, health, cost, latency) for rapid per-request scoring. The full 14-factor engine incorporates additional heuristics such as geographic proximity, model capability alignment, and historical provider reliability trends. The I4 mode serves as the default for model: "auto" requests where computational efficiency is prioritized, while the 14-factor mode can be activated for complex routing scenarios requiring deeper analysis.

How does OmniRoute normalize latency and cost values for scoring?

The system applies dedicated normalization functions—normalizeLatency() for millisecond values and normalizeCost() for USD-per-1K-token pricing—to compress these metrics into standardized 0-1 ranges. This normalization occurs within calculateScore() before weight application, ensuring that high-cost providers receive lower scores and low-latency providers receive higher scores regardless of the absolute numerical ranges of their raw data.

Can I customize the provider scoring weights in OmniRoute?

Yes, the scoring system accepts custom ScoringWeights objects that override the DEFAULT_WEIGHTS constant. You can pass a custom weight configuration to the scorePool() function to adjust the relative importance of quota, health, cost, and latency factors. This allows you to optimize routing for specific use cases, such as favoring lower costs during batch processing or prioritizing health scores during peak traffic periods.

What happens if the highest-scored provider fails during a request?

The Auto-Combo Engine implements automatic fallback logic that preserves the ranked order established by the I4-factor scoring. If the primary provider connection fails, the system immediately retries the request with the second-ranked candidate, then the third, and so on until success or exhaustion of the candidate pool. This ensures that the scoring investment made during candidate selection continues to guide resilience behavior throughout the request lifecycle.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}