How to Configure Custom Combo Routing Strategies in OmniRoute for Specific Use Cases

OmniRoute's combo routing system lets you direct a single request to multiple LLM providers through pluggable strategies like priority, weighted, round-robin, and fusion, configured via three cascading layers of defaults, provider overrides, and per-request settings.

OmniRoute's combo feature is one of its most powerful capabilities, allowing a single API request to be intelligently distributed across multiple LLM providers or models. According to the diegosouzapw/OmniRoute source code, the routing behavior is governed by configurable strategies that you can customize for latency optimization, cost reduction, reliability, or quality. This guide explains exactly how to configure these strategies for your specific use cases.

Understanding the Configuration Layers

OmniRoute resolves combo configuration through three cascading layers, merged by resolveComboConfig() in open-sse/services/comboConfig.ts (lines 60-87):

Layer	Source	Priority
Global defaults	DEFAULT_COMBO_CONFIG in open-sse/services/comboConfig.ts	Base values for all combos
Provider overrides	settings.comboDefaults → settings.providerOverrides[provider]	Applied when targeting specific providers
Per-combo config	combo.config in the request payload	Highest priority, wins over other layers

The merge logic uses object spreading to flatten these layers, strip legacy keys, and return a resolved configuration consumed by the combo engine in open-sse/services/combo.ts.

Available Routing Strategies

The combo engine supports extensive strategy options, enumerated at the top of open-sse/services/combo.ts:

// open-sse/services/combo.ts (excerpt)
/**
 * Supports: priority, weighted, round-robin, random, least-used,
 * cost-optimized, reset-aware, reset-window, strict-random, auto,
 * fill-first, p2c, lkgp, context-optimized, context-relay, and fusion
 */

The config.strategy value determines which handler the engine dispatches to:

• priority — Try targets in order, fall back on failure
• weighted — Distribute load by configured weights
• round-robin — Cycle through targets sequentially
• fusion — Aggregate responses from multiple models
• auto — Dynamically select candidates using 13-factor scoring
• cost-optimized — Prioritize lowest-cost providers
• reset-aware / reset-window — Handle context window resets intelligently

Where to Declare Your Custom Strategy

Method 1: Per-Request Payload (Fastest)

Add a config object directly to your combo request for one-off customization:

{
  "model": "combo",
  "combo": {
    "name": "my-priority-combo",
    "strategy": "priority",
    "models": ["openai/gpt-4o", "gemini/flash", "anthropic/claude-3.5-sonnet"],
    "config": {
      "maxRetries": 2,
      "retryDelayMs": 1500,
      "fallbackDelayMs": 300,
      "responseValidation": {
        "type": "json-schema",
        "schema": {
          "required": ["choices"],
          "properties": { "choices": { "type": "array" } }
        }
      }
    }
  },
  "messages": [
    { "role": "user", "content": "Explain quantum tunnelling in one paragraph." }
  ]
}

This tries gpt-4o first, retries twice with 1.5s delays, then falls back through gemini and anthropic if validation fails.

Method 2: Provider-Level Overrides (Settings Table)

Store organization-wide defaults in the database for persistent policy:

INSERT INTO settings (key, value) VALUES
  ('comboDefaults', '{"strategy":"weighted","maxRetries":1,"retryDelayMs":2000}'),
  ('providerOverrides', '{"gemini":{"strategy":"cost-optimized","queueDepth":10}}')
ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value;

All combos default to weighted strategy with single retry, while gemini-targeting combos use cost-optimized with queue depth capped at 10.

Method 3: Global Defaults (Code Change)

Modify open-sse/services/comboConfig.ts for deployment-wide changes:

// open-sse/services/comboConfig.ts
export const DEFAULT_COMBO_CONFIG = {
  // ... other defaults
  strategy: "reset-aware",
  predictiveTtftMs: 1500,
  zeroLatencyOptimizationsEnabled: true,
  // ...
};

Rebuild and redeploy to affect all new combos.

Fine-Tuning Strategy Behavior

Each strategy respects optional knobs defined in DEFAULT_COMBO_CONFIG (lines 96-155 of comboConfig.ts):

Config Key	Purpose	When to Use
maxRetries / retryDelayMs	Retry failing targets	Unstable provider connections
fallbackDelayMs	Wait before next target	Preserve low-latency primary
queueDepth / queueTimeoutMs	Semaphore queue limits	Prevent pile-up in round-robin
handOffModel / handOffProviders	Post-response model escalation	"Warm up" to stronger models
responseValidation	Schema/predicate validation	Failover on low-quality outputs
zeroLatencyOptimizationsEnabled	Predictive TTFT circuit breaker	Latency-sensitive APIs
comboTimeoutMs	Global wall-clock ceiling	Prevent runaway combo loops
shadowRouting / evalRouting	Background A/B testing	Experiment safely

These are automatically honored by phaseComboSetup() → handleComboChat() in combo.ts (lines 64-92).

How Configuration Flows Through the System

1. API entry: src/app/api/v1/chat/completions/route.ts receives the payload
2. Setup phase: phaseComboSetup() calls resolveComboSetupConfig() → resolveComboConfig() to flatten all layers
3. Strategy dispatch: The resolved strategy selects the handler:
   • round-robin → handleRoundRobinCombo() (~line 90)
   • fusion → tryFusionDispatch() (~line 104)
   • auto → buildAutoCandidates() (~line 1000)
   • Others → generic target-iteration loop (~line 1045)

All paths respect resolveComboTargetTimeoutMsForCombo() (lines 69-82) and the global config.comboTimeoutMs (lines 1414-1416).

Key Source Files Reference

File	Function	Direct Link
open-sse/services/comboConfig.ts	Defaults, merging, timeout helpers	comboConfig.ts
open-sse/services/combo.ts	Main engine, strategy dispatch	combo.ts
open-sse/services/combo/comboSetup.ts	Config resolution, session injection	comboSetup.ts
open-sse/services/combo/comboStructure.ts	Target list resolution	comboStructure.ts
src/app/api/v1/chat/completions/route.ts	API surface	route.ts

Summary

• Three-layer config: Defaults → provider overrides → per-request combo.config, merged by resolveComboConfig()
• 15+ strategies: From simple priority fallbacks to sophisticated auto scoring and fusion aggregation
• Three declaration methods: Request payload for one-offs, settings table for policies, code changes for global defaults
• Fine-grained control: Retries, timeouts, validation, hand-offs, and circuit breakers all configurable per strategy

Frequently Asked Questions

What is the fastest way to test a different routing strategy?

Add a config object with your desired strategy directly to the request payload. This bypasses all defaults and provider overrides, letting you experiment without database changes or redeployment.

How do I make all Gemini combos use cost-optimization while keeping other providers on priority?

Insert a provider override in the settings table: providerOverrides with {"gemini":{"strategy":"cost-optimized"}}. The resolveComboConfig() function checks provider-specific overrides after defaults but before per-request config, so Gemini-targeting combos will automatically use your specified strategy.

Why does my combo configuration not match what I sent in the request?

Check that your payload uses the correct structure: combo.config (not top-level config). Also verify no provider overrides are active for your target providers. Enable debug logging to see the resolved configuration output from phaseComboSetup().

What happens if I specify a strategy that doesn't exist?

The combo engine defaults to safe behavior, typically falling back to priority or rejecting the request with a validation error depending on your OmniRoute version. The valid strategies are explicitly listed in the header comment of open-sse/services/combo.ts — reference this when configuring custom strategies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}