How to Configure Combo Routing with Multiple Providers and Strategies in OmniRoute

OmniRoute lets you define named combos that group multiple AI providers and models, then applies configurable strategies like weighted load balancing or latency-aware auto-routing to select the optimal target for each request.

Configuring combo routing with multiple providers and strategies allows you to build resilient, cost-effective AI infrastructure. OmniRoute’s combo system stores configurations in a SQLite-backed combos table and executes routing logic through a dedicated service layer that supports over a dozen selection strategies.

Understanding Combo Routing

A combo is a named collection of provider-model pairs stored with optional weights, limits, and metadata. When a request arrives, OmniRoute resolves the combo definition, applies the configured strategy to order targets, and iterates through them until success or exhaustion.

The core routing engine lives in open-sse/services/combo.ts, where the entry comment documents all supported strategies: priority, weighted, round-robin, random, least-used, cost-optimized, reset-aware, auto, fill-first, p2c, lkgp, context-optimized, context-relay, and fusion. The execution flow follows three steps:

1. Resolution: resolveComboConfig and resolveComboTargets fetch and validate the combo definition.
2. Strategy Application: applyStrategyOrdering and resolveAutoStrategyOrder reorder targets based on the selected strategy.
3. Execution: The engine invokes handleSingleModel for each target in sequence until a response succeeds or all targets fail.

Creating a Multi-Provider Combo

Via the REST API

Create combos dynamically by sending a POST request to /api/v1/combos/[token], defined in src/app/api/v1/combos/[token]/[[...slug]]/route.ts. The endpoint accepts a JSON payload validated against Zod schemas in open-sse/handlers/.

{
  "name": "my-multi-provider-combo",
  "providers": [
    { "provider": "openai", "model": "gpt-4o-mini", "weight": 1 },
    { "provider": "anthropic", "model": "claude-3-sonnet-100k", "weight": 2 }
  ],
  "strategy": "auto",
  "fallback": "auto-fallback"
}

Set strategy to any supported strategy name. Include fallback to specify a secondary combo name used if all primary targets fail (handled by attemptCompatRejectedFallback).

Programmatically via the Database

For custom tooling or seed scripts, import the database helper from src/lib/db/combos.ts:

import { createCombo } from "../../src/lib/db/combos.ts";

await createCombo({
  name: "my-multi-provider-combo",
  providers: [
    { provider: "openai", model: "gpt-4o-mini", weight: 1 },
    { provider: "anthropic", model: "claude-3-sonnet-100k", weight: 2 }
  ],
  strategy: "auto",
  fallback: "auto-fallback"
});

The createCombo function persists the configuration to the SQLite combos table and validates against the Combo schema defined in the same file.

Choosing and Configuring Strategies

The strategy field determines how targets are ordered. The implementation in open-sse/services/combo/applyStrategyOrdering.ts dispatches to specific resolvers based on the string value:

• priority: Uses targets in the exact order defined (primary → secondary).
• weighted: Shuffles targets proportionally to assigned weight values.
• round-robin: Cycles through targets evenly using per-combo counters (rrCounters) defined in open-sse/services/combo.ts.
• random: Applies fisherYatesShuffle for stochastic selection.
• least-used: Queries recent usage stats from src/lib/usageDb to prefer idle providers.
• cost-optimized: Scores targets via scoreAutoTargets to minimize monetary cost while respecting quotas.
• auto: Calls buildAutoCandidates to dynamically rank targets by latency, quota availability, and task weight, then expands the candidate pool with expandAutoComboCandidatePool.
• fusion: Combines multiple scoring signals through handleFusionChat in fusion.ts.
• reset-aware: Respects provider-specific rate-limit reset windows using RESET_WINDOW_NAMES and resolveResetWindowConfig.

To change strategies, update the strategy field via the API or database helper; the routing engine automatically switches the resolver function.

Advanced Combo Features

Session Stickiness

Enable sticky sessions to bind a conversation to the same provider for its lifetime. The system tracks bindings via applySessionStickiness and persists them through recordStickyBinding.

Quota Pre-flight

Before routing, buildAutoCandidates can discard low-quota connections when quotaPreflight.enabled is true. Configure this in resolveResilienceSettings to prevent routing to exhausted providers.

Context-Aware Routing

Use context-optimized or context-relay strategies when working with large contexts. These check getKnownContextOverflow to ensure the selected provider has sufficient token budget for the request payload.

Fallback Combos

Define resilience chains by setting the fallback property to another combo name. If all targets in the primary combo fail, the engine automatically switches to the fallback configuration without returning an error to the client.

Summary

• A combo groups provider-model pairs with weights and strategy configuration in the SQLite combos table.
• Create combos via POST /api/v1/combos/[token] or programmatically using createCombo in src/lib/db/combos.ts.
• Choose from 14 strategies including weighted, round-robin, auto, and fusion, implemented in open-sse/services/combo/applyStrategyOrdering.ts.
• Activate advanced features like session stickiness, quota pre-flight, and fallback combos for production resilience.
• Route requests by sending the combo name in the X-Omni-Combo header or combo query parameter.

Frequently Asked Questions

How do I switch from weighted to latency-based routing?

Update the combo's strategy field to "auto". The resolveAutoStrategyOrder function in open-sse/services/combo.ts automatically collects latency metrics via buildAutoCandidates and reorders targets based on real-time performance. Persist the change via the REST API or by calling the database update method in src/lib/db/combos.ts.

Can I combine different models from OpenAI and Anthropic in one combo?

Yes. Include multiple provider entries in the providers array, each specifying the provider name and model identifier. OmniRoute treats these as distinct targets within the same combo, allowing strategies like round-robin or weighted to distribute load across vendors.

What happens if all providers in a combo fail?

If every target exhausts its retry budget, the engine checks for a fallback combo name. If defined, it recursively resolves the fallback combo via attemptCompatRejectedFallback. Without a fallback, the request returns an error to the client.

Where is the combo configuration stored?

Combo definitions persist in a SQLite database table named combos. The src/lib/db/combos.ts file defines the schema and CRUD operations, while open-sse/services/combo.ts contains the runtime resolution logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}