How OmniRoute Auto-Combo Scoring Works with Live Evaluation

OmniRoute's auto-combo engine calculates a weighted score across 13 provider factors—such as quota, latency, cost, and health—and continuously refines those scores through live evaluation via the Self-Healing component to optimize routing decisions in real time.

OmniRoute is an open-source AI gateway that dynamically routes requests across multiple LLM providers using an intelligent scoring system. The auto-combo scoring mechanism, implemented in the diegosouzapw/OmniRoute repository, combines static weighted factors with runtime telemetry to select the optimal provider-model pair for every request. This article breaks down the scoring algorithm, weight normalization, and real-time evaluation pipeline based on the source code in the release/v3.8.49 branch.

The 13-Factor Scoring Model

At the core of the routing decision is the ProviderCandidate interface defined in [open-sse/services/autoCombo/scoring.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/autoCombo/scoring.ts). The engine evaluates every candidate against 13 distinct signals to compute a normalized score between 0 and 1:

• Quota – Remaining quota percentage (higher is better).
• Health – Provider health rating derived from recent success rates.
• CostInv – Inverse of cost per million tokens (cheaper providers score higher).
• LatencyInv – Inverse of p95 latency in milliseconds (faster providers score higher).
• TaskFit – Compatibility between the model's capabilities and the request's task type.
• Stability – Historical error-rate stability.
• TierPriority – Account-tier priority boosts (Ultra > Pro > Free).
• TierAffinity – Additional tier-based affinity weighting.
• SpecificityMatch – Exact model-name or version match bonus.
• ContextAffinity – Suitability for the request's required context length.
• CacheAffinity – Advantage for prompt-caching hits (optional).
• ResetWindowAffinity – Preference for accounts nearing their quota reset window.
• ConnectionDensity – Distribution metric to prevent overloading a single account.

These factors are aggregated using a weighted sum where each factor is multiplied by its corresponding weight and normalized.

Weight Configuration and Normalization

The system ships with a default 12-factor weight set defined as DEFAULT_WEIGHTS in [scoring.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/autoCombo/scoring.ts). This default vector sums to exactly 1.0, ensuring deterministic scoring out of the box.

Teams can override these defaults through the UI configuration. Any custom weight set is passed through normalizeScoringWeights(), which rescales the inputs so the total always equals 1. This prevents user error from distorting the scoring scale and maintains mathematical consistency across evaluations.

Live Evaluation via Self-Healing

Static scoring determines the initial candidate ranking, but live evaluation adapts these scores based on real-world performance. After each request completes, the Self-Healing component in [open-sse/services/autoCombo/selfHealing.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/autoCombo/selfHealing.ts) executes healer.evaluate(provider, observedScore, circuitState).

The evaluation logic consumes live telemetry including errorRate, failureRate, avgTtftMs (time to first token), and avgE2ELatencyMs. If a provider exhibits elevated error rates or latency degradation, the health factor is penalized, lowering its score for subsequent requests. Conversely, consistently fast, successful responses boost health and latency factors. This feedback loop ensures the routing engine learns from recent provider behavior without requiring manual intervention.

The updated health metrics are then fed back into the next scoring pass inside [open-sse/services/combo.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/combo.ts), specifically within resolveComboTargets(), which rebuilds the candidate list using the refreshed factors.

Selection and Routing Strategy

Once scores are computed and health-adjusted, the routing logic in [open-sse/services/combo/autoStrategy.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/combo/autoStrategy.ts) sorts candidates by final score. The highest-scoring provider that also satisfies circuit-breaker constraints (state must be CLOSED) and quota requirements is selected for the request.

Code Implementation Example

The following TypeScript demonstrates how to construct candidates, normalize custom weights, and compute scores using the same logic found in the OmniRoute source:

import {
  ProviderCandidate,
  DEFAULT_WEIGHTS,
  normalizeScoringWeights,
} from '@omniroute/open-sse/services/autoCombo/scoring';

// 1. Define provider candidates (normally fetched from the database)
const candidates: ProviderCandidate[] = [
  {
    provider: 'openai',
    model: 'gpt-4o',
    quotaRemaining: 80,
    quotaTotal: 100,
    circuitBreakerState: 'CLOSED',
    costPer1MTokens: 0.03,
    p95LatencyMs: 150,
    latencyStdDev: 20,
    errorRate: 0.01,
  },
  {
    provider: 'anthropic',
    model: 'claude-3-sonnet',
    quotaRemaining: 60,
    quotaTotal: 100,
    circuitBreakerState: 'CLOSED',
    costPer1MTokens: 0.025,
    p95LatencyMs: 120,
    latencyStdDev: 15,
    errorRate: 0.005,
  },
];

// 2. Load and normalize weights (UI overrides merged with defaults)
const customWeights = { latencyInv: 0.2, costInv: 0.1 };
const weights = normalizeScoringWeights(customWeights); // Sum guaranteed = 1

// 3. Compute weighted score for a candidate
function computeScore(c: ProviderCandidate, w: typeof DEFAULT_WEIGHTS): number {
  const factors = {
    quota: c.quotaRemaining / 100,
    health: 1 - (c.errorRate ?? 0),
    costInv: 1 / c.costPer1MTokens,
    latencyInv: 1 / c.p95LatencyMs,
    taskFit: 1, // Simplified for demo
    stability: 1 - (c.errorRate ?? 0),
    tierPriority: 1,
    tierAffinity: 1,
    specificityMatch: 1,
    contextAffinity: 1,
    cacheAffinity: 0,
    resetWindowAffinity: 0,
    connectionDensity: 1,
  };

  return Object.entries(w).reduce(
    (sum, [key, weight]) => sum + (factors[key as keyof typeof factors] ?? 0) * weight,
    0,
  );
}

// 4. Score and rank all candidates
const ranked = candidates
  .map(c => ({ candidate: c, score: computeScore(c, weights) }))
  .sort((a, b) => b.score - a.score);

console.log('Best provider:', ranked[0].candidate.provider, ranked[0].score);

Live evaluation occurs asynchronously after the request finishes; the Self-Healing engine updates the provider's internal health metrics, which automatically influence the next call to computeScore.

Debugging the Scoring Decision

To audit why a specific provider was chosen, OmniRoute exposes the Combo Scoring Inspector endpoint implemented in [src/lib/usage/comboScoringInspector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/src/lib/usage/comboScoringInspector.ts). This utility returns the raw factor values, applied weights, and individual score contributions for every candidate evaluated during a request, enabling operators to verify the routing logic in production.

Summary

• OmniRoute evaluates every provider against 13 weighted factors including quota, cost, latency, and health.
• Weights are normalized via normalizeScoringWeights() to maintain a 0–1 scoring scale.
• The Self-Healing engine in selfHealing.ts performs live evaluation after each request, adjusting health scores based on observed error rates and latency.
• Final routing decisions combine static scoring with dynamic health updates to select the optimal provider.
• The Combo Scoring Inspector provides full visibility into scoring calculations for debugging.

Frequently Asked Questions

What are the 13 factors used in OmniRoute's auto-combo scoring?

The 13 factors are quota remaining, health rating, inverse cost, inverse latency, task fit, stability, tier priority, tier affinity, specificity match, context affinity, cache affinity, reset window affinity, and connection density. Each factor is defined in the ProviderCandidate interface within [open-sse/services/autoCombo/scoring.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/autoCombo/scoring.ts).

How does the Self-Healing component update provider scores?

After each request, healer.evaluate() in [selfHealing.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/autoCombo/selfHealing.ts) processes live metrics such as errorRate and avgE2ELatencyMs. It penalizes the health factor for providers showing degradation and boosts it for consistent performance, directly affecting the next scoring iteration.

Can I customize the scoring weights for my deployment?

Yes. You can override the DEFAULT_WEIGHTS through the UI configuration. The system merges custom values with defaults and passes them through normalizeScoringWeights() to ensure the total sum remains 1.0, preventing scoring distortions.

Where can I inspect the scoring breakdown for a specific request?

Use the Combo Scoring Inspector endpoint defined in [src/lib/usage/comboScoringInspector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/src/lib/usage/comboScoringInspector.ts). It returns the raw factor values, weight assignments, and computed contributions for every candidate considered during routing, allowing full transparency into the decision process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}