# OmniRoute Scoring Engine: The 13 Candidate Evaluation Criteria Explained

> Discover the 13 candidate evaluation criteria of the OmniRoute scoring engine. Understand how quota, health, cost, latency, and affinity factors determine provider scores.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-06

---

**The OmniRoute auto-combo scoring engine evaluates provider candidates using 13 normalized telemetry factors—quota, health, cost, latency, task fit, stability, tier priority, tier affinity, specificity match, context affinity, cache affinity, reset window affinity, and connection density—that are weighted and summed to produce a final score between 0 and 1.**

OmniRoute is an open-source routing layer for LLM and AI service providers. Its **auto-combo routing engine**, implemented in [`open-sse/services/autoCombo/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoring.ts), automatically selects the optimal provider for each request by scoring candidates against a multi-dimensional evaluation framework. This article breaks down every candidate evaluation criterion, how each is computed from raw telemetry, and how the final ranking is determined.

## How the Scoring Pipeline Works

The scoring process follows three distinct stages as implemented in the source code:

1. **Telemetry collection** — Gather raw metrics for each candidate provider (quota remaining, circuit-breaker state, latency statistics, etc.)
2. **Factor normalization** — Convert each metric into a normalized **factor** in the range [0, 1] using the `calculateFactors` function
3. **Weighted aggregation** — Apply configurable **scoring weights** via `calculateScore` to produce a final ranking value, clamped to [0, 1]

All 13 evaluation criteria are defined by the `ScoringFactors` interface and computed within `calculateFactors` (lines 20–37).

## The 13 Candidate Evaluation Criteria

Each criterion is derived from provider telemetry and normalized before entering the weighted sum. Here's the complete breakdown:

### Resource Availability Factors

| Criterion | Derivation | Source Lines |
|-----------|------------|--------------|
| **quota** | `candidate.quotaRemaining / 100` — percentage of remaining quota for the account | 20–21 |
| **health** | Circuit-breaker state: `CLOSED = 1`, `HALF_OPEN = 0.5`, otherwise `0` | 22–26 |
| **connectionDensity** | Normalized connection pool size: `(size - 1) / 10` | 37 |

### Cost and Performance Factors

| Criterion | Derivation | Source Lines |
|-----------|------------|--------------|
| **costInv** | Inverted cost: `1 - candidate.costPer1MTokens / maxCost` | 27 |
| **latencyInv** | Inverted P95 latency: `1 - candidate.p95LatencyMs / maxLatency` | 28 |
| **stability** | Inverted latency variance: `1 - candidate.latencyStdDev / maxStdDev` | 30 |

### Task and Model Alignment Factors

| Criterion | Derivation | Source Lines |
|-----------|------------|--------------|
| **taskFit** | Model-to-task fitness from pluggable `getTaskFitness()` function | 29 |
| **tierPriority** | Normalized tier score (Ultra/Pro/Standard/Free) plus reset-interval bonus via `calculateTierScore` | 31 |
| **tierAffinity** | Alignment with manifest's *recommended-min-tier* via `calculateTierAffinity` | 32 |
| **specificityMatch** | Match against manifest's *specificity* score via `calculateSpecificityMatch` | 33 |

### Session and Cache Affinity Factors

| Criterion | Derivation | Source Lines |
|-----------|------------|--------------|
| **contextAffinity** | Re-use preference for current session's provider/account/model; defaults to `0.5` | 34 |
| **cacheAffinity** | Preference for cached prompt-key hits; defaults to `0` | 35 |
| **resetWindowAffinity** | Preference for providers with nearer quota-reset windows; defaults to `0.5` | 36 |

Each factor is **individually clamped** to [0, 1] by `clamp01` (lines 15–18) to prevent malformed telemetry from corrupting the final score.

## Default Weight Configuration

The `DEFAULT_WEIGHTS` object in [`scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/scoring.ts) establishes the relative importance of each criterion:

```typescript
export const DEFAULT_WEIGHTS: ScoringWeights = {
  quota: 0.15,
  health: 0.20,
  costInv: 0.15,
  latencyInv: 0.12,
  taskFit: 0.08,
  stability: 0.05,
  tierPriority: 0.05,
  tierAffinity: 0.05,
  specificityMatch: 0.05,
  contextAffinity: 0.05,
  cacheAffinity: 0,          // disabled by default
  resetWindowAffinity: 0,    // disabled by default
  connectionDensity: 0.05,
};

```

Notable design decisions in the default configuration:

- **Health (0.20)** and **quota (0.15)** receive highest priority — reliability and capacity dominate routing decisions
- **Cache affinity** and **reset window affinity** are zeroed out unless explicitly enabled
- Weights are automatically normalized to sum to 1 by `normalizeScoringWeights` (lines 59–76)

## Calculating the Final Score

The `calculateScore` function performs a weighted sum of all factors, with defensive null-coalescing for optional weights:

```typescript
export function calculateScore(factors: ScoringFactors, weights: ScoringWeights): number {
  return clamp01(
    weights.quota * factors.quota +
    weights.health * factors.health +
    weights.costInv * factors.costInv +
    weights.latencyInv * factors.latencyInv +
    weights.taskFit * factors.taskFit +
    weights.stability * factors.stability +
    weights.tierPriority * factors.tierPriority +
    (weights.tierAffinity ?? 0) * factors.tierAffinity +
    (weights.specificityMatch ?? 0) * factors.specificityMatch +
    (weights.contextAffinity ?? 0) * factors.contextAffinity +
    (weights.cacheAffinity ?? 0) * (factors.cacheAffinity ?? 0) +
    (weights.resetWindowAffinity ?? 0) * factors.resetWindowAffinity +
    (weights.connectionDensity ?? 0) * factors.connectionDensity
  );
}

```

The clamped result (0–1) directly determines candidate ranking. Higher scores indicate more suitable providers for the current request.

## Practical Usage Examples

### Scoring a Candidate Pool

Use `scorePool()` to rank providers for a specific task:

```typescript
// open-sse/services/autoCombo/scoring.ts
const scored = scorePool(
  providerCandidates,    // ProviderCandidate[]
  "chat.completions",    // task type
  customWeights,         // optional ScoringWeights override
  modelFitnessFn,        // optional task-fit function
  manifestRoutingHint    // optional manifest hint
);
// Returns: ScoredProvider[] sorted by descending score

```

### Customizing Weights Per Request

Override default weights through the `autoConfig.weights` object in your combo request:

```json
{
  "autoConfig": {
    "weights": {
      "quota": 0.30,
      "health": 0.25,
      "costInv": 0.20,
      "latencyInv": 0.15,
      "taskFit": 0.10,
      "stability": 0,
      "tierPriority": 0,
      "tierAffinity": 0,
      "specificityMatch": 0,
      "contextAffinity": 0,
      "cacheAffinity": 0,
      "resetWindowAffinity": 0,
      "connectionDensity": 0
    }
  }
}

```

The `parseAutoConfig` routine in [`open-sse/services/combo/autoConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/autoConfig.ts) processes this configuration, normalizes the weights, and passes them to `scorePool`.

### Extending with Custom Factors

Add new evaluation criteria by extending the core interfaces:

```typescript
// Step 1: Extend interfaces
interface ScoringFactors { 
  reliability: number;
  // ... existing fields
}

interface ScoringWeights { 
  reliability: number;
  // ... existing fields
}

// Step 2: Add to calculateScore
weights.reliability * factors.reliability

// Step 3: Populate in calculateFactors
reliability: clamp01(candidate.reliabilityMetric ?? 0.5)

```

Update `DEFAULT_WEIGHTS` with your new factor's default value. All downstream code, including the test suite in [`tests/unit/auto-combo-scoring-clamp.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/auto-combo-scoring-clamp.test.ts), will recognize the extension.

## Key Source Files

| File | Purpose |
|------|---------|
| [`open-sse/services/autoCombo/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoring.ts) | Core scoring logic: `ScoringFactors`, `ScoringWeights`, `calculateFactors`, `calculateScore`, `DEFAULT_WEIGHTS` |
| [`open-sse/services/combo/autoConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/autoConfig.ts) | Parses per-combo weight overrides and routing configuration |
| [`open-sse/services/combo/targetResolution.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/targetResolution.ts) | Orchestrates candidate resolution and calls `scorePool` |
| [`src/lib/usage/comboScoringInspector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/usage/comboScoringInspector.ts) | Debug/inspection utilities for scoring outcomes |
| [`tests/unit/auto-combo-scoring-clamp.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/auto-combo-scoring-clamp.test.ts) | Validates factor clamping and weight normalization |

## Summary

- **13 normalized factors** constitute OmniRoute's complete candidate evaluation criteria: quota, health, costInv, latencyInv, taskFit, stability, tierPriority, tierAffinity, specificityMatch, contextAffinity, cacheAffinity, resetWindowAffinity, and connectionDensity
- Each factor is **derived from live telemetry** and clamped to [0, 1] before weighting
- **Configurable weights** default to health-first (0.20) with cost and quota as secondary priorities (0.15 each)
- The scoring pipeline is **fully extensible** — new factors can be added by updating interfaces and the calculation functions
- All critical logic resides in [`open-sse/services/autoCombo/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/autoCombo/scoring.ts) with per-request customization supported through `autoConfig.weights`

## Frequently Asked Questions

### How does OmniRoute handle providers with zero remaining quota?

The **quota** factor evaluates to 0 when `candidate.quotaRemaining` is 0, effectively eliminating that provider from selection unless other factors and weight adjustments compensate. Since quota carries a 0.15 default weight, a zero-quota provider faces severe penalization in the final ranking.

### Can I disable specific evaluation criteria entirely?

Yes. Set any weight to `0` in your custom `ScoringWeights` configuration. The `calculateScore` function uses null-coalescing (`?? 0`) for optional weights, so omitted or zero-valued criteria contribute nothing to the final sum. This is how `cacheAffinity` and `resetWindowAffinity` are disabled by default.

### What happens if telemetry data is missing or malformed?

All factors pass through `clamp01` during calculation in `calculateFactors` (lines 15–18). This guarantees every input is bounded to [0, 1] regardless of source data quality. The defensive design prevents a single corrupted metric from destabilizing the entire routing decision.

### How does OmniRoute balance cost versus latency in routing decisions?

By default, **costInv (0.15)** and **latencyInv (0.12)** are weighted similarly but not identically. Cost receives slight precedence. You can invert this priority by supplying custom weights — for latency-sensitive workloads, increase `latencyInv` and reduce `costInv` proportionally.