How to Configure OmniRoute Auto-Combo Engine with 12-Factor Scoring

TLDR: Configure OmniRoute's Auto-Combo engine by defining a weights object in your Combo Runtime Config that assigns values to 12 individual scoring factors—six primary health and cost metrics plus six tier and context affinity metrics—allowing the engine to dynamically rank LLM candidates per request.

OmniRoute is an open-source LLM routing platform that dynamically selects and balances providers through its Auto-Combo engine. The engine uses a weighted 12-factor scoring model to evaluate candidates across health, cost, latency, and contextual affinity dimensions. This configuration is controlled via the comboRuntimeConfigSchema defined in src/shared/validation/schemas/combo.ts and processed by the scoring inspector in src/lib/usage/comboScoringInspector.ts.

Understanding the 12-Factor Scoring Model

The OmniRoute Auto-Combo engine organizes its decision logic into two logical factor groups (sometimes referenced as I2-factor scoring), comprising 12 individual weighted metrics:

Primary Health and Cost Metrics (Group 1)

• quota: Remaining quota minus quota penalties
• health: Recent success rate and latency performance
• costInv: Inverse cost (lower cost receives higher scores)
• latencyInv: Inverse latency (faster response receives higher scores)
• taskFit: Model suitability for the specific requested task
• stability: Output consistency and reliability ratings

Tier and Context Affinity Metrics (Group 2)

• tierPriority: Preference for higher-tier providers
• tierAffinity: Historical success with the same provider tier
• specificityMatch: Alignment with request-specific tags
• contextAffinity: Similarity to past successful conversation contexts
• cacheAffinity: Potential for cached response reuse
• resetWindowAffinity: Compliance with recent reset-window limits

Each factor accepts a numeric weight between 0 and 1. Any omitted weight defaults to 0, effectively disabling that factor from the scoring calculation.

Configuration Schema and Validation

The scoring weights are declared in the Combo Runtime Config schema located at src/shared/validation/schemas/combo.ts. This schema validates the weights object when you create or update a combo configuration.

The validation logic ensures that:

• All 12 factor keys are optional
• Provided weights must be numeric values
• Omitted factors default to 0 and are excluded from the weighted calculation

When processing requests, the engine builds a scoring inspector payload via src/lib/usage/comboScoringInspector.ts, multiplying each raw factor value by its configured weight to compute the final candidate score.

Step-by-Step Configuration

Follow these steps to enable 12-factor scoring for your Auto-Combo deployment.

1. Define the Combo Configuration

Create a JSON payload that specifies your candidate pool and assigns weights to the desired factors. The strategy field must be set to "auto" to activate the Auto-Combo engine.

{
  "name": "production-12-factor",
  "strategy": "auto",
  "candidatePool": [
    "openai-gpt-4",
    "anthropic-claude-2",
    "gemini-1.5-flash"
  ],
  "weights": {
    "quota": 0.20,
    "health": 0.25,
    "costInv": 0.15,
    "latencyInv": 0.10,
    "taskFit": 0.10,
    "stability": 0.10,
    "tierPriority": 0.05,
    "tierAffinity": 0.05,
    "specificityMatch": 0.05,
    "contextAffinity": 0.08,
    "cacheAffinity": 0,
    "resetWindowAffinity": 0
  }
}

2. Submit via the REST API

Send a POST request to the combo creation endpoint. The route validates your payload against comboRuntimeConfigSchema before persisting the configuration.

const comboConfig = {
  name: "production-12-factor",
  strategy: "auto",
  candidatePool: ["openai-gpt-4", "azure-gpt-35-turbo"],
  weights: {
    quota: 0.2,
    health: 0.25,
    costInv: 0.15,
    latencyInv: 0.1,
    taskFit: 0.1,
    stability: 0.1,
    tierPriority: 0.05,
    tierAffinity: 0.05,
    specificityMatch: 0.05,
    contextAffinity: 0.08,
    cacheAffinity: 0,
    resetWindowAffinity: 0
  }
};

await fetch("http://localhost:20128/api/v1/combo", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(comboConfig)
});

3. Activate the Combo

Reference your configured combo in API requests using the combo query parameter, or set it as the default routing strategy in src/app/api/v1/chat/completions/route.ts for automatic application to all chat completion requests.

Debugging with the Scoring Inspector

The Scoring Inspector endpoint exposes raw factor calculations and final weighted scores for debugging and tuning purposes. Access this via src/app/api/usage/combo-scoring-inspector/route.ts.

Query the inspector to verify your 12-factor weights are calculating as expected:

curl http://localhost:20128/api/usage/combo-scoring-inspector?combo=production-12-factor

The response includes the raw factor vector (actual metric values collected per candidate) and the computed score after applying your configured weights. Use this data to adjust weight distributions between primary health metrics and affinity-based factors according to your SLA requirements.

Key Implementation Files

• src/shared/validation/schemas/combo.ts: Defines the Combo schema including all 12 scoring factor weights
• src/lib/usage/comboScoringInspector.ts: Builds inspection payloads showing weighted score calculations
• src/app/api/usage/combo-scoring-inspector/route.ts: HTTP endpoint for scoring diagnostics
• docs/routing/AUTO-COMBO.md: Comprehensive documentation of the Auto-Combo engine and scoring model

Summary

• OmniRoute's Auto-Combo engine evaluates LLM candidates using 12 configurable scoring factors organized into two logical groups.
• Configure weights in the Combo Runtime Config via the weights object, with omitted factors defaulting to 0.
• Primary factors (quota, health, cost, latency, task fit, stability) balance infrastructure health and economics.
• Affinity factors (tier priority, context matching, caching) optimize for historical performance and contextual suitability.
• Validate configurations using the Scoring Inspector endpoint at /api/usage/combo-scoring-inspector.

Frequently Asked Questions

What happens if I omit certain factors from the weights object?

Omitted factors default to a weight of 0 and are excluded from the scoring calculation. The engine only evaluates factors with non-zero weights, allowing you to run simplified configurations using subsets of the 12-factor model without modifying the underlying schema.

How do I prioritize cost efficiency over latency in the scoring?

Increase the costInv weight and decrease the latencyInv weight in your configuration. For example, set "costInv": 0.30 and "latencyInv": 0.05 to strongly prefer cheaper providers while tolerating higher latency, or set latencyInv to 0 to ignore speed entirely.

Can I use the Scoring Inspector without activating the combo?

Yes. The Scoring Inspector endpoint (GET /api/usage/combo-scoring-inspector) accepts a combo parameter and returns calculated scores for all candidates in the pool without routing actual traffic. This allows safe testing of weight configurations before production deployment.

What's the difference between tierPriority and tierAffinity?

tierPriority assigns static preference to higher-tier providers regardless of history, while tierAffinity scores providers based on their historical success rate specifically with the same tier classification as the current request. Use tierPriority for SLA guarantees and tierAffinity for optimizing based on empirical performance patterns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}