# How OmniRoute Evaluates LLM Providers in Real-Time Using 14-Factor Auto-Combo Scoring

> Discover how OmniRoute's 14-factor Auto-Combo scoring evaluates LLM providers in real-time. Learn about live telemetry, weighted scores, and normalized rankings for optimal performance.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-16

---

**OmniRoute's Auto-Combo engine computes a real-time weighted score for every LLM provider candidate by combining live telemetry—including quota, circuit-breaker health, cost, latency, stability, and contextual affinity—into a single normalized ranking from 0 to 1.**

The **Auto-Combo scoring system** is the core intelligence behind OmniRoute's ability to route requests dynamically. Instead of relying on static rules, it evaluates every available provider-model pair at request time using 14 weighted factors. This article breaks down how the scoring algorithm works, where it lives in the codebase, and how you can customize it for your workload.

## Where the Scoring Logic Lives

The scoring implementation is concentrated in two files:

- **Core algorithm**: [[`open-sse/services/autoCombo/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoring.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/autoCombo/scoring.ts) — contains `calculateFactors`, `calculateScore`, and `scorePool`
- **Documentation**: [[`docs/routing/AUTO-COMBO.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/routing/AUTO-COMBO.md)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/docs/routing/AUTO-COMBO.md) — explains the 14-factor weighting system and mode packs

All scoring functions are pure TypeScript utilities with no external dependencies beyond the tier resolver.

## Step 1: Gathering Live Telemetry with ProviderCandidate

Before any scoring happens, OmniRoute builds a **ProviderCandidate** object for every available provider-model pair. This happens at request time, so the data reflects the current state of each connection.

The `ProviderCandidate` interface (defined in [`scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/scoring.ts)) captures:

| Field | Purpose |
|-------|---------|
| `quotaRemaining` | Tokens or requests left in current window |
| `circuitBreakerState` | Closed, half-open, or open (for resilience) |
| `costPer1MTokens` | Pricing for cost-optimization |
| `p95LatencyMs` | Measured tail latency |
| `latencyStdDev` | Stability indicator (lower is better) |
| `getTaskFitness` | Plug-in function for model-task matching |
| `tier` / `quotaResetInterval` | Account-level priority handling |
| Affinity hints | Context, cache, session, reset-window, connection density |

This telemetry is fetched fresh for every routing decision, making the **14-factor Auto-Combo scoring system** truly real-time.

## Step 2: Normalizing Pool-Wide Maxima

The `calculateFactors` function first computes inverse baselines across the candidate pool:

```typescript
const maxCost = Math.max(...candidates.map(c => c.costPer1MTokens));
const maxLatency = Math.max(...candidates.map(c => c.p95LatencyMs));
const maxStdDev = Math.max(...candidates.map(c => c.latencyStdDev));

```

These maxima enable **relative scoring**: a candidate's cost or latency is judged against the worst performer in the pool, not against absolute thresholds. This keeps scores portable across different deployment environments.

## Step 3: Deriving the 14 Scoring Factors

Each candidate is transformed into a `ScoringFactors` object with all values clamped to `[0, 1]` via `clamp01`:

| Factor | Calculation | Meaning |
|--------|-------------|---------|
| `quota` | `quotaRemaining / 100` | More quota remaining = higher score |
| `health` | `1.0` (closed), `0.5` (half-open), `0` (open) | Circuit-breaker state |
| `costInv` | `1 - (cost / maxCost)` | Cheaper providers score higher |
| `latencyInv` | `1 - (latency / maxLatency)` | Faster providers score higher |
| `stability` | `1 - (stdDev / maxStdDev)` | More consistent latency = higher score |
| `taskFit` | `getTaskFitness(model, taskType)` | Model-task alignment from plug-in |
| `tierPriority` | `calculateTierScore(tier, resetInterval)` | Account tier preference |
| `tierAffinity` | Optional affinity boost | Tier-to-tier routing preference |
| `specificityMatch` | Optional match score | Exact model matching |
| `contextAffinity` | Optional state score | Sticky context routing |
| `cacheAffinity` | Optional hit rate | Prompt cache benefit |
| `sessionAvailability` | Optional availability | Session continuity |
| `resetWindowAffinity` | Optional temporal score | Quota reset timing |
| `connectionDensity` | Optional load balance | Connection distribution |

Factor extraction is handled by `calculateFactors` in the scoring module. All optional affinity factors default to neutral values (typically `0.5`) when not provided.

## Step 4: Applying Configurable Weights

The **14-factor Auto-Combo scoring system** uses `DEFAULT_WEIGHTS` as a baseline, but callers can supply custom weights via the `ScoringWeights` interface:

```typescript
export const DEFAULT_WEIGHTS: ScoringWeights = {
  quota: 0.10,
  health: 0.15,
  costInv: 0.15,
  latencyInv: 0.15,
  taskFit: 0.15,
  stability: 0.10,
  tierPriority: 0.05,
  tierAffinity: 0.05,
  specificityMatch: 0.03,
  contextAffinity: 0.03,
  cacheAffinity: 0.02,
  sessionAvailability: 0.02,
  resetWindowAffinity: 0.02,
  connectionDensity: 0.03,
};

```

The `validateWeights` helper ensures weights sum to approximately 1. For UI-driven customizations, `normalizeScoringWeights` rescales any input to a proper distribution.

## Step 5: Computing the Final Score

The `calculateScore` function performs a weighted sum:

```typescript
export function calculateScore(
  factors: ScoringFactors,
  weights: ScoringWeights
): number {
  const score = Object.keys(weights).reduce((sum, key) => {
    return sum + (factors[key] ?? 0.5) * (weights[key] ?? 0);
  }, 0);
  return clamp01(score);
}

```

The result is clamped to `[0, 1]` and represents the candidate's **overall suitability** for the current request.

## Step 6: Selecting the Best Provider with scorePool

The `scorePool` function orchestrates the entire pipeline:

```typescript
export function scorePool(
  candidates: ProviderCandidate[],
  taskType: string,
  weights: ScoringWeights = DEFAULT_WEIGHTS,
  taskFitness: TaskFitnessFn = defaultTaskFitness,
  manifestHint: ManifestHint | null = null
): ScoredProvider[] {
  const maxima = computePoolMaxima(candidates);
  
  return candidates
    .map(candidate => {
      const factors = calculateFactors(candidate, maxima, taskType, taskFitness, manifestHint);
      const score = calculateScore(factors, weights);
      return { ...candidate, score, factors };
    })
    .sort((a, b) => b.score - a.score);
}

```

The returned `ScoredProvider[]` is sorted by descending score. The first element is the optimal choice for the request.

## Practical Example: Customizing Weights for Latency-Sensitive Workloads

```typescript
import {
  scorePool,
  DEFAULT_WEIGHTS,
  ScoringWeights,
} from '@omniroute/open-sse/services/autoCombo/scoring';

// Retrieve live candidates from the connection registry
const candidates = await getProviderCandidates(); // → ProviderCandidate[]

// Define task-specific fitness: boost code-generation models
function taskFitness(model: string, task: string): number {
  return task === 'coding' && model.includes('codex') ? 0.9 : 0.5;
}

// Customize weights for latency-critical routing
const fastWeights: ScoringWeights = {
  ...DEFAULT_WEIGHTS,
  latencyInv: 0.25,    // prioritize speed
  costInv: 0.10,       // de-emphasize cost
  stability: 0.15,     // ensure consistency
};

const scored = scorePool(
  candidates,
  'coding',
  fastWeights,
  taskFitness,
  null
);

const best = scored[0];
console.log(`Selected ${best.provider}/${best.model} (score=${best.score.toFixed(3)})`);
// Example output: Selected azure/gpt-4 (score=0.847)

```

## How Real-Time Evaluation Enables Dynamic Routing

Because **all 14 factors are computed from live telemetry**, OmniRoute can adapt instantly to:

- **Quota exhaustion** — providers with depleted budgets automatically score lower via `quota` and `tierPriority`
- **Circuit breaker trips** — unhealthy providers score near-zero via `health`
- **Latency spikes** — degraded providers lose ground via `latencyInv` and `stability`
- **Context affinity** — sticky routing via `contextAffinity` and `sessionAvailability`
- **Cache optimization** — prompt cache hits boost `cacheAffinity`

This makes the **14-factor Auto-Combo scoring system** suitable for production workloads requiring sub-second failover without manual intervention.

## Key Files and Functions Reference

| Component | Location | Role |
|-----------|----------|------|
| `scorePool` | [`open-sse/services/autoCombo/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoring.ts) | Main entry point for candidate ranking |
| `calculateFactors` | Same file | Transforms telemetry into normalized factors |
| `calculateScore` | Same file | Weighted sum with clamping |
| `ProviderCandidate` interface | Same file | Telemetry schema for candidates |
| `DEFAULT_WEIGHTS` | Same file | Baseline 14-factor weight distribution |
| `validateWeights` / `normalizeScoringWeights` | Same file | Weight validation and rescaling |
| `calculateTierScore` | Imported from `../tierResolver` | Account tier scoring |
| `classifyTier` | `../tierResolver` | Tier classification utility |
| User documentation | [`docs/routing/AUTO-COMBO.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/routing/AUTO-COMBO.md) | Conceptual guide and mode packs |

## Summary

- **Real-time telemetry** — `ProviderCandidate` collects quota, health, cost, latency, and affinity data fresh for every request
- **14 normalized factors** — `calculateFactors` produces values in `[0,1]` including inverses for cost/latency and plug-in task fitness
- **Configurable weights** — `DEFAULT_WEIGHTS` covers all 14 factors, customizable via `ScoringWeights` with validation
- **Weighted scoring** — `calculateScore` combines factors into a single suitability score
- **Ranked selection** — `scorePool` returns candidates sorted by score, enabling automatic optimal selection

The scoring system is stateless, pure, and fast enough to run on every request without caching, ensuring routing decisions always reflect current conditions.

## Frequently Asked Questions

### What are the 14 factors in OmniRoute's Auto-Combo scoring?

The 14 factors are: `quota`, `health`, `costInv`, `latencyInv`, `taskFit`, `stability`, `tierPriority`, `tierAffinity`, `specificityMatch`, `contextAffinity`, `cacheAffinity`, `sessionAvailability`, `resetWindowAffinity`, and `connectionDensity`. They cover resource availability, performance, cost, reliability, account priority, and contextual routing preferences.

### How does OmniRoute handle providers with depleted quotas?

The `quota` factor directly incorporates `quotaRemaining`, while `tierPriority` and `resetWindowAffinity` consider account tier and quota reset timing. A provider with zero remaining quota scores near-zero on `quota`, causing it to drop in the rankings unless other weights are extreme.

### Can I create custom weight profiles for different use cases?

Yes. Pass a custom `ScoringWeights` object to `scorePool`. The `validateWeights` function ensures your weights sum to ~1, and `normalizeScoringWeights` can rescale UI inputs. Common profiles include "fast" (high `latencyInv`), "cheap" (high `costInv`), and "resilient" (high `health` and `stability`).

### Why does OmniRoute use inverse scoring for cost and latency?

Cost and latency are **minimization objectives** (lower is better), while scores must maximize toward 1. The formulas `1 - (value / maxValue)` flip these so that cheaper/faster candidates score higher, keeping all factors directionally consistent.

### How does the scoring system respond to circuit breaker state changes?

The `health` factor reads `circuitBreakerState` directly: closed circuits score 1.0, half-open score 0.5, and open circuits score 0.0. Because telemetry is fetched at request time, a tripped breaker immediately zeros out the provider's health contribution, removing it from contention.