How to Configure Custom Combo Routing Strategies in OmniRoute

You configure custom combo routing strategies in OmniRoute through three cascading layers: global defaults in comboConfig.ts, provider-specific overrides in the settings table, and per-request config in the payload's combo.config object, with final resolution handled by resolveComboConfig().

OmniRoute's combo routing feature allows a single request to intelligently distribute traffic across multiple LLM providers and models. The routing behavior is controlled by pluggable strategies—from simple priority fallbacks to sophisticated cost-optimized and context-aware routing. This guide explains how to configure these strategies using the actual source code implementation in diegosouzapw/OmniRoute.

Understanding the Configuration Hierarchy

OmniRoute resolves combo configuration through three layers defined in open-sse/services/comboConfig.ts. The resolveComboConfig() function (lines 60-87) merges these layers using object spreading, with later layers overriding earlier ones:

Layer	Source	Priority
Global defaults	DEFAULT_COMBO_CONFIG in comboConfig.ts	Lowest
Provider overrides	settings.comboDefaults and settings.providerOverrides[provider]	Medium
Per-combo config	combo.config in the request payload	Highest

This cascading design lets you set organization-wide defaults while still allowing fine-grained control for individual requests.

Available Routing Strategies

The combo engine in open-sse/services/combo.ts supports extensive strategy options. The supported strategies are enumerated at the top of that file:

priority, weighted, round-robin, random, least-used,
cost-optimized, reset-aware, reset-window, strict-random, auto,
fill-first, p2c, lkgp, context-optimized, context-relay, and fusion

The engine dispatches to specific handlers based on config.strategy. For example:

• round-robin → handleRoundRobinCombo()
• fusion → tryFusionDispatch()
• auto → buildAutoCandidates()

Configuration Methods

Per-Request Configuration (Highest Priority)

The fastest way to customize routing for a single request is adding a config object to the payload's combo field:

{
  "model": "combo",
  "combo": {
    "name": "my-priority-combo",
    "strategy": "priority",
    "models": ["openai/gpt-4o", "gemini/flash", "anthropic/claude-3.5-sonnet"],
    "config": {
      "maxRetries": 2,
      "retryDelayMs": 1500,
      "fallbackDelayMs": 300,
      "responseValidation": {
        "type": "json-schema",
        "schema": {
          "required": ["choices"],
          "properties": {
            "choices": { "type": "array" }
          }
        }
      }
    }
  },
  "messages": [
    { "role": "user", "content": "Explain quantum tunnelling in one paragraph." }
  ]
}

This configuration tries gpt-4o first, retries twice with 1.5s delays on failure, then falls back to gemini/flash. If gemini returns HTTP 200 but fails JSON schema validation, the engine proceeds to anthropic/claude-3.5-sonnet.

Provider-Level Overrides

For persistent defaults tied to specific providers, use the settings table. This is ideal when different providers need different strategies:

INSERT INTO settings (key, value) VALUES
  ('comboDefaults', '{"strategy":"weighted","maxRetries":1,"retryDelayMs":2000}'),
  ('providerOverrides', '{"gemini":{"strategy":"cost-optimized","queueDepth":10}}')
ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value;

This sets all combos to weighted routing by default, while any combo targeting gemini uses cost-optimized strategy with a queue depth limit of 10.

Global Configuration Changes

To change defaults for all new combos, modify DEFAULT_COMBO_CONFIG directly:

// open-sse/services/comboConfig.ts
export const DEFAULT_COMBO_CONFIG = {
  // ... other defaults ...
  strategy: "reset-aware",
  predictiveTtftMs: 1500,
  // ...
};

Rebuild and redeploy to apply changes across your OmniRoute instance.

Strategy Tuning Parameters

DEFAULT_COMBO_CONFIG (lines 96-155 in comboConfig.ts) defines optional knobs that all strategies respect:

• maxRetries / retryDelayMs — Retry count and backoff for failing targets
• fallbackDelayMs — Wait time before falling back to next target
• queueDepth / queueTimeoutMs — Semaphore limits for round-robin and weighted strategies
• handOffModel / handOffProviders — Post-response model hand-off for warm-up patterns
• responseValidation — Predicate for validating 200 responses (JSON schema, etc.)
• zeroLatencyOptimizationsEnabled — Predictive TTFT circuit breaker
• comboTimeoutMs — Global wall-clock timeout for the entire combo
• shadowRouting / evalRouting — Background routing for A/B testing

These parameters are automatically honored by the combo engine after resolveComboConfig() flattens the configuration layers.

Code Execution Flow

Understanding the execution path helps debug custom configurations:

1. Request entry — src/app/api/v1/chat/completions/route.ts forwards combo requests to handleComboChat() (lines 64-92 in combo.ts)

2. Configuration resolution — phaseComboSetup() calls resolveComboSetupConfig() → resolveComboConfig() to merge the three configuration layers

3. Strategy dispatch — The resolved config.strategy value drives execution through specific handlers in combo.ts

4. Timeout enforcement — Both per-target timeouts (resolveComboTargetTimeoutMsForCombo(), lines 69-82) and global combo timeouts (config.comboTimeoutMs, lines 1414-1416) are applied

Key Source Files

File	Purpose
open-sse/services/comboConfig.ts	Default settings, merge logic, timeout helpers
open-sse/services/combo.ts	Main engine: strategy parsing, target iteration, retries
open-sse/services/combo/comboSetup.ts	Pre-setup phase and resolved config injection
open-sse/services/combo/comboStructure.ts	Target list resolution and provider wildcard expansion
src/app/api/v1/chat/completions/route.ts	API surface receiving JSON payloads

Summary

• Three-layer configuration: global defaults → provider overrides → per-request combo.config
• Strategy selection: Set via config.strategy; options include priority, weighted, cost-optimized, fusion, and 11 others
• Resolution function: resolveComboConfig() in comboConfig.ts flattens all layers
• Execution entry: handleComboChat() dispatches to strategy-specific handlers
• Fine-tuning: All strategies respect retry, timeout, validation, and hand-off parameters from resolved config

Frequently Asked Questions

What is the fastest way to test a new combo routing strategy?

Add a config object with your desired strategy directly to the request payload's combo field. This per-request configuration has highest priority and requires no database changes or redeployment.

How do I make all Gemini combos use cost-optimized routing while keeping other providers on priority?

Insert a provider override into the settings table: providerOverrides.gemini = {"strategy":"cost-optimized"}. This medium-priority layer automatically applies to any combo targeting Gemini, without affecting OpenAI, Anthropic, or other providers.

Where does OmniRoute enforce timeouts for custom combo strategies?

Timeout logic lives in two places: resolveComboTargetTimeoutMsForCombo() (lines 69-82 in comboConfig.ts) calculates per-target timeouts based on the resolved configuration, while config.comboTimeoutMs provides a global ceiling enforced by the main combo handler around lines 1414-1416 in combo.ts.

Can I validate response quality before accepting a combo result?

Yes. Set responseValidation in your combo configuration with a JSON schema or custom predicate. The engine will treat validation failures as trigger conditions for fallback to the next target model, even when the HTTP response is 200.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}