How the `latencyInv` Factor Influences Auto-Combo Selection in OmniRoute

The latencyInv factor is an inverse-normalized p95 latency score that contributes approximately 11% to the Auto-Combo ranking algorithm, prioritizing faster provider-model pairs while maintaining balance with health, cost, and quota signals.

The latencyInv factor plays a critical role in the intelligent routing decisions of OmniRoute, an open-source AI gateway that dynamically selects optimal provider-model combinations. Understanding how this inverse latency metric influences the Auto-Combo scoring system allows developers to fine-tune their routing strategies for latency-sensitive applications. The factor works by converting raw p95 latency measurements into a normalized score where faster providers receive values closer to 1.0 and slower providers approach 0.0.

What Is the latencyInv Factor?

latencyInv represents the inverse-normalized p95 latency used by the Auto-Combo scorer to prefer fast, reliable provider-model pairs. Rather than using raw millisecond values, OmniRoute normalizes latency against the worst-performing candidate in the current pool. This relative scoring approach ensures the algorithm remains robust when new providers are added or when underlying infrastructure performance fluctuates.

The factor is one of nine primary scoring dimensions in the Auto-Combo system, alongside quota, health, costInv, taskFit, stability, tierPriority, tierAffinity, and specificityMatch. By default, latencyInv carries a weight of 0.1143 (approximately 11.43%), ensuring latency matters but does not override critical signals like provider health.

How latencyInv Is Calculated

The calculation occurs in two distinct phases within open-sse/services/autoCombo/scoring.ts: pool-wide maximum determination and per-candidate inverse normalization.

Pool-Wide Maximum Latency Normalization

First, OmniRoute establishes the upper bound for latency in the current candidate pool. The computePoolMaxima() function scans all available providers to find the highest p95LatencyMs value:

// From open-sse/services/autoCombo/scoring.ts lines 46-53
function computePoolMaxima(candidates: ProviderCandidate[]) {
  let maxLatency = 0;
  for (const c of candidates) {
    if (c.p95LatencyMs > maxLatency) {
      maxLatency = c.p95LatencyMs;
    }
  }
  return { maxLatency };
}

This maximum value serves as the denominator for all subsequent inverse calculations, ensuring scores reflect relative performance rather than absolute milliseconds.

Inverse Latency Formula

For each candidate, the system calculates latencyInv using a clamped inverse ratio:

// From open-sse/services/autoCombo/scoring.ts lines 80-83
const latencyInv = clamp01(1 - candidate.p95LatencyMs / maxLatency);

The clamp01() utility ensures the result stays within the 0.0 to 1.0 range. A provider with p95 latency near the pool maximum receives a factor approaching 0.0, while the fastest provider (theoretical 0ms latency) would receive 1.0. In practice, the provider with the lowest latency in the pool receives the highest latencyInv score.

Weighting and Score Contribution

The latencyInv factor integrates into the final ranking through a weighted sum calculation during the calculateScore() execution.

Default Weight Configuration

The default weight constant is defined in the scoring module:

// From open-sse/services/autoCombo/scoring.ts lines 53-58
export const DEFAULT_WEIGHTS = {
  quota: 0.20,
  health: 0.25,
  costInv: 0.15,
  latencyInv: 0.1143,
  taskFit: 0.10,
  stability: 0.05,
  tierPriority: 0.05,
  tierAffinity: 0.05,
  specificityMatch: 0.0857,
};

This approximately 11.43% allocation ensures that latency influences selection without allowing extremely fast but unhealthy providers to dominate the rankings.

Interaction with Other Scoring Factors

During calculateScore(), the weighted contribution follows this pattern:

// Simplified from open-sse/services/autoCombo/scoring.ts lines 49-53
const score = 
  weights.quota * factors.quota +
  weights.health * factors.health +
  weights.costInv * factors.costInv +
  weights.latencyInv * factors.latencyInv +
  // ... other factors

The scorePool() function then sorts the resulting ScoredProvider array in descending order by total score. Because latencyInv is multiplied by its weight before addition, providers with poor latency see their overall scores reduced proportionally, while fast providers gain a competitive advantage.

Customizing the latencyInv Weight

Developers can override the default weight by supplying custom combo configurations through the UI or programmatic API. Increasing the weight transforms the strategy into a "fast-first" selector, while decreasing it shifts priority toward cost optimization or quota management.

Here is an example configuration that boosts latency importance to 35%:

{
  "id": "latency-prioritized",
  "name": "Low Latency Auto",
  "strategy": "auto",
  "config": {
    "auto": {
      "candidatePool": ["anthropic", "google", "openai"],
      "weights": {
        "quota": 0.15,
        "health": 0.30,
        "costInv": 0.05,
        "latencyInv": 0.35,
        "taskFit": 0.10,
        "stability": 0.00,
        "tierPriority": 0.05
      }
    }
  }
}

When using the TypeScript API, normalize custom weights before passing them to scorePool():

import { normalizeScoringWeights, scorePool } from '@/open-sse/services/autoCombo/scoring';

const customWeights = {
  quota: 0.15,
  health: 0.25,
  costInv: 0.05,
  latencyInv: 0.35, // Prioritize speed over cost
  taskFit: 0.10,
  stability: 0.00,
  tierPriority: 0.10,
};

const normalized = normalizeScoringWeights(customWeights);
const scored = scorePool(candidatePool, 'chat', normalized, taskFitFn);

Practical Implementation Examples

Extracting latencyInv for a Candidate

To inspect the raw factor value for debugging or monitoring:

import { computePoolMaxima, calculateFactors } from '@/open-sse/services/autoCombo/scoring';

const candidate = {
  provider: 'openai',
  model: 'gpt-4o',
  p95LatencyMs: 420,
  latencyStdDev: 30,
  // ... other required telemetry fields
};

const pool = [candidate /*, other candidates */];
const { maxLatency } = computePoolMaxima(pool);
const factors = calculateFactors(candidate, pool, 'chat', () => 0.5);

console.log(`latencyInv: ${factors.latencyInv.toFixed(3)}`);
// → latencyInv: 0.160 (when maxLatency = 500ms: 1 - 420/500 = 0.16)

Full Auto-Combo Scoring Pipeline

A complete implementation showing how latencyInv influences the final selection:

import { scorePool, normalizeScoringWeights } from '@/open-sse/services/autoCombo/scoring';

// 1. Gather telemetry from all candidate providers
const pool = await fetchProviderTelemetry(); // Returns ProviderCandidate[]

// 2. Apply custom weights emphasizing latency
const weights = normalizeScoringWeights({
  health: 0.20,
  latencyInv: 0.30, // Boost latency importance
  costInv: 0.10,
  quota: 0.15,
  taskFit: 0.25
});

// 3. Score and rank the pool
const scoredProviders = scorePool(
  pool,
  'code-generation', // Task type
  weights,
  (model, task) => calculateTaskFit(model, task) // Custom task-fit function
);

// 4. Select the top-ranked provider
const bestCandidate = scoredProviders[0];
console.log(
  `Selected: ${bestCandidate.provider}/${bestCandidate.model} ` +
  `(score: ${bestCandidate.score.toFixed(3)}, ` +
  `latencyInv: ${bestCandidate.factors.latencyInv.toFixed(3)})`
);

Summary

• Inverse normalization: latencyInv uses 1 - (candidate.p95LatencyMs / poolMaxLatency) to convert absolute latency into a relative 0.0-1.0 score.
• Default influence: With a weight of 0.1143 (11.43%), latency impacts ranking without overwhelming health or quota signals.
• Pool-relative scoring: The factor adjusts automatically as provider pools change, ensuring consistent relative performance evaluation.
• Configurable priority: Developers can increase the weight up to 1.0 for latency-critical applications or reduce it to 0.0 to ignore speed entirely.
• Source location: All calculation logic resides in open-sse/services/autoCombo/scoring.ts, specifically within computePoolMaxima() and calculateScore().

Frequently Asked Questions

What does latencyInv stand for in OmniRoute?

latencyInv stands for "inverse latency." It represents a normalized score where higher values indicate faster provider response times. The factor is calculated as 1 - (p95Latency / maxPoolLatency), meaning providers with p95 latency closer to the pool's slowest candidate receive scores near 0.0, while the fastest providers approach 1.0.

How does latencyInv interact with the health factor?

The health factor typically carries a higher default weight (0.25) than latencyInv (0.1143). This hierarchy ensures that a provider with excellent latency but failing health checks (circuit breaker OPEN) will still rank lower than a healthy provider with moderate latency. The weighted sum approach in calculateScore() balances these competing signals to prevent selecting fast but unreliable endpoints.

Can I disable latency-based scoring entirely?

Yes. You can disable the latencyInv factor by setting its weight to 0.0 in your custom configuration:

const weights = normalizeScoringWeights({
  latencyInv: 0.0,
  health: 0.30,
  quota: 0.30,
  costInv: 0.20,
  taskFit: 0.20
});

When disabled, the Auto-Combo strategy will ignore p95 latency data and select providers based solely on the remaining weighted factors.

Where is the latencyInv calculation defined in the source code?

The calculation is defined in open-sse/services/autoCombo/scoring.ts within the calculateFactors() function (lines 80-83). The normalization depends on computePoolMaxima() (lines 46-53) to establish the pool-wide maximum latency. Default weights are exported as DEFAULT_WEIGHTS (lines 53-58), and the final scoring integration occurs in calculateScore() (lines 49-53) and scorePool() (lines 99-124).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}