How to Configure Custom Combo Routing with Specific Provider Chains in OmniRoute

Configure custom combo routing in OmniRoute by defining a config object with handoffProviders for the provider chain, selecting a strategy (priority, round-robin, or fusion), and tuning timeouts, retries, and queue depth—merged across global defaults, provider overrides, and per-combo settings.

OmniRoute's combo routing engine lets you stitch together ordered provider chains that execute with defined failover logic. This article walks through the configuration cascade, key parameters, and practical implementations based on the OmniRoute source code.

Understanding the Configuration Cascade

Effective combo configuration in OmniRoute emerges from four layered sources, merged by resolveComboConfig() in open-sse/services/comboConfig.ts:

1. Global defaults – DEFAULT_COMBO_CONFIG defines system-wide baselines
2. Application-wide overrides – settings.comboDefaults stored in the local DB
3. Provider-specific overrides – settings.providerOverrides[provider] affecting all combos using that provider
4. Per-combo configuration – the config object supplied when creating a combo

The cascade discards undefined or null values and legacy resilience keys, ensuring the most specific setting wins.

Key Routing Strategy Options

The strategy field determines how OmniRoute traverses your provider chain:

Strategy	Behavior	Use Case
priority	Try providers in handoffProviders order until success	Guaranteed fallback ordering
round-robin	Distribute requests evenly across providers	Load balancing
fusion	Aggregate responses from multiple providers	Ensemble results

Strategy normalization occurs in src/shared/constants/routingStrategies.ts via normalizeRoutingStrategy().

Building a Custom Provider Chain

Essential Configuration Parameters

Define your chain through the config object when creating a combo:

• handoffProviders – Array of provider names in desired order
• handoffThreshold – Confidence threshold triggering handoff (0.0–1.0)
• maxRetries – Retry attempts per target before cascading
• targetTimeoutMs – Per-provider timeout (subject to upstream ceiling)
• queueDepth – Max requests in semaphore queue (0 disables queuing)
• retryDelayMs – Delay between retry attempts

Example: Priority Chain with Three Providers

{
  "name": "my-custom-chain",
  "strategy": "priority",
  "config": {
    "maxRetries": 2,
    "targetTimeoutMs": 90000,
    "handoffProviders": ["openai", "anthropic", "gemini"],
    "handoffThreshold": 0.9,
    "queueDepth": 10,
    "contextRequirements": {
      "minContextWindow": 500,
      "maxContextWindow": 4000,
      "preferLargeContext": true,
      "contextFilterMode": "strict"
    }
  }
}

The handoffProviders array establishes strict precedence: OmniRoute attempts OpenAI first, falls back to Anthropic on failure, then Gemini.

Setting Provider-Level Overrides

Apply settings across all combos using a specific provider through settings.providerOverrides:

{
  "comboDefaults": null,
  "providerOverrides": {
    "openai": {
      "queueDepth": 15,
      "handoffThreshold": 0.8
    },
    "anthropic": {
      "queueDepth": 12
    }
  }
}

These persist in src/lib/db/settings.ts and merge before per-combo configuration.

Runtime Configuration Resolution

When a request hits /v1/chat/completions, phaseComboSetup() in open-sse/services/combo/comboSetup.ts executes:

1. Calls resolveComboSetupConfig() to build effective configuration
2. Computes comboTargetTimeoutMs via resolveComboTargetTimeoutMsForCombo() – applying safe floors based on upstream timeout and cooldown-wait budget
3. Determines comboTimeoutMs for overall combo execution
4. Resolves comboQueueDepth through resolveComboQueueDepth()

Targets dispatch via handleSingleModel() in open-sse/services/combo.ts, respecting per-target timeouts and cooldown eligibility (isComboCooldownWaitEligible).

Implementation Examples

CLI Creation

omniroute combo create \
  --name my-custom-chain \
  --strategy priority \
  --config '{
    "maxRetries":2,
    "targetTimeoutMs":90000,
    "handoffProviders":["openai","anthropic","gemini"],
    "handoffThreshold":0.9,
    "queueDepth":10
  }'

API Endpoint

POST /v1/combo

{
  "name": "my-custom-chain",
  "strategy": "priority",
  "config": {
    "maxRetries": 2,
    "targetTimeoutMs": 90000,
    "handoffProviders": ["openai", "anthropic", "gemini"],
    "handoffThreshold": 0.9,
    "queueDepth": 10
  }
}

Updating Provider Defaults

omniroute settings set providerOverrides='{
  "openai": {"queueDepth":15,"handoffThreshold":0.8},
  "anthropic": {"queueDepth":12}
}'

Advanced: Timeout and Queue Mechanics

Per-Target Timeout Calculation

resolveComboTargetTimeoutMsForCombo() in comboConfig.ts computes final timeout by:

• Respecting explicit targetTimeoutMs if provided
• Applying upstream timeout ceiling
• Subtracting cooldown-wait budget when isComboCooldownWaitEligible applies

Queue Depth Behavior

queueDepth Value	Effect
0	Queuing disabled; immediate cascade on saturation
1+	Requests queue up to limit before triggering next provider
null/undefined	Falls back to DEFAULT_COMBO_CONFIG.queueDepth

Configuration Files Reference

File	Function	Key Exports
open-sse/services/comboConfig.ts	Configuration merging and resolution	resolveComboConfig(), resolveComboTargetTimeoutMs(), resolveComboQueueDepth(), DEFAULT_COMBO_CONFIG
open-sse/services/combo/comboSetup.ts	Runtime combo initialization	phaseComboSetup(), resolveComboSetupConfig()
open-sse/services/combo.ts	Execution and dispatch	handleComboChat(), handleSingleModel()
src/shared/constants/routingStrategies.ts	Strategy validation	normalizeRoutingStrategy()
src/lib/resilience/settings.ts	Resilience parameters	Cooldown-wait eligibility, retry policies
bin/cli/commands/combo.mjs	CLI interface	Combo CRUD operations

Summary

• Four-layer cascade determines final combo configuration: global → application → provider → per-combo
• handoffProviders array defines explicit provider ordering for priority chains
• strategy selection (priority/round-robin/fusion) controls traversal and failover logic
• Timeouts and queue depth resolve through dedicated functions that enforce safety floors and upstream limits
• Provider overrides enable cross-cutting configuration without touching individual combos

Frequently Asked Questions

How does OmniRoute choose which provider to use first in a combo?

OmniRoute uses the strategy field to determine ordering. With priority strategy, providers execute in the exact sequence of the handoffProviders array. Other strategies like round-robin rotate starting positions for load distribution. The normalized strategy is validated by normalizeRoutingStrategy() in src/shared/constants/routingStrategies.ts.

Can I set different timeouts for different providers in the same combo?

While targetTimeoutMs is defined at the combo level, you can achieve provider-specific timeouts through the cascade: set application-wide defaults in comboDefaults, then override per provider in settings.providerOverrides[provider]. The most specific value wins when resolveComboConfig() merges layers.

What happens when all providers in a chain fail?

The combo's maxRetries controls per-target retries. After exhausting retries on all providers, handleComboChat() returns the final error. You can increase reliability by adding more providers to handoffProviders or increasing maxRetries, balanced against comboTimeoutMs to prevent excessive latency.

Where are combo configurations stored?

Per-combo configurations reside in OmniRoute's database, accessible via API or CLI. Provider overrides and application defaults store in settings (managed through src/lib/db/settings.ts). Global defaults compile into DEFAULT_COMBO_CONFIG in open-sse/services/comboConfig.ts as TypeScript constants.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}