What Are OmniRoute Combos and How Do They Work: Multi-Model Routing Explained

OmniRoute combos are named collections of one or more provider-model targets that use a configurable strategy to automatically select, fallback between, and synthesize responses from multiple AI models.

OmniRoute combos provide a declarative, strategy-driven routing layer within the diegosouzapw/OmniRoute project. According to the source code, they transparently handle provider selection, quota management, error fallback, and distributed tracing for any multi-model request.

OmniRoute Combo Structure and Configuration

A combo definition lives in the database (src/lib/db/combos.ts) and acts as a portable routing policy that can be referenced by name in API requests.

Core Combo Properties

Each combo specifies three critical components:

• Strategy – Determines execution order and fallback behavior (e.g., priority, weighted, fusion, auto). The canonical list is maintained in src/shared/constants/routingStrategies.ts.
• Target List – An ordered array where each target contains a provider identifier, model string, optional connection ID, and per-target configuration such as timeout values and quota-share flags.
• Config – Global settings including maxRetries, comboTimeoutMs, and retry policies.

{
  "name": "gpt-4-plus-auto",
  "strategy": "auto",
  "targets": [
    { "provider": "openai", "modelStr": "gpt-4o-mini", "connectionId": "conn-1" },
    { "provider": "anthropic", "modelStr": "claude-3-5-sonnet", "connectionId": "conn-2" }
  ],
  "config": { "maxRetries": 2, "comboTimeoutMs": 12000 }
}

How the Combo Routing Engine Executes Requests

When a request is routed to a combo, the engine in open-sse/services/combo.ts orchestrates a multi-stage execution pipeline.

Target Resolution

The engine first expands the combo into an ordered array of ResolvedComboTarget objects using the resolveComboTargets function found in open-sse/services/combo/comboStructure.ts. This step validates the target list and prepares the execution DAG.

Auto-Candidate Building

For strategies like auto, the engine enriches the candidate list using buildAutoCandidates (lines 56-99 in combo.ts). This function scores each target by:

• Latency – Historical response time metrics.
• Cost – Token pricing per provider-model pair.
• Quota availability – Remaining token or request budgets.
• Circuit-breaker state – Whether the provider is currently healthy.
• Context affinity – Session stickiness for stateful conversations.

The Execution Loop

The core iteration logic resides in handleComboChatInner (the internal implementation of handleComboChat). As implemented in open-sse/services/combo.ts, the engine:

1. Iterates through the ordered targets or candidate pool.
2. Invokes the user-provided handleSingleModel function for each model.
3. Returns immediately upon the first successful response.
4. Exhausts all targets before failing if no successful response is obtained.

Resilience Checks and Error Handling

During iteration, the combo engine applies a comprehensive set of resilience filters to avoid unhealthy providers.

Health and Quota Validation

The engine checks four critical conditions before attempting a target:

• Circuit-breaker status – Uses getCircuitBreaker to skip providers whose breaker is OPEN.
• Provider-wide cooldown – Respects global cooldown windows via isProviderInCooldown.
• Model lockout – Avoids temporarily disabled models using isModelLocked from the quota subsystem.
• Quota-exhaustion cutoff – Blocks targets below a threshold via resolveQuotaExhaustionCutoffForTarget (lines 149-166).

Retry and Diagnostics

When a target fails, the engine records detailed diagnostics using buildComboDiag, tracking pool size, attempted count, excluded targets, and attempt order. The system supports three retry modes:

• maxSetRetries – Retry the entire target set from the beginning.
• maxRetries – Per-target retry attempts.
• Cooldown-aware retry – Uses dispatchWithCooldownRetry (lines 238-262) to wait for provider cooldown windows before reattempting.

Fusion and Pipeline Strategies

Standard combos execute targets sequentially, but the fusion and pipeline strategies operate differently. As implemented in open-sse/services/combo/dispatchPrelude.ts, the tryFusionDispatch function fans out requests to multiple models in parallel and synthesizes a unified final response from the individual model outputs.

This enables use cases like:

• Ensembling multiple models for higher accuracy.
• Aggregating specialized models for multi-modal comprehension.
• Running A/B tests across provider responses.

Observability and Request Tracing

Every combo execution generates a trace identifier that is injected into the response headers. When a combo succeeds, the response includes the X-OmniRoute-Combo-Trace header. Operators can look up the exact decision sequence using getComboTrace, which returns the step-by-step routing decisions, including which targets were attempted, skipped, or succeeded.

// Dispatch a chat request using the combo router
import { handleComboChat } from '@/open-sse/services/combo';

const response = await handleComboChat({
  body: incomingPayload,
  combo: dbComboObject,
  handleSingleModel: async (body, model) => fetchProviderResponse(body, model),
  log: logger,
  settings: {},
  allCombos: [],
});

// Inspect the execution trace
const traceId = response.headers.get('X-OmniRoute-Combo-Trace');
const trace = getComboTrace(traceId!);
console.log(trace.decisions); 
// [{ step: "gpt-4o-mini", decision: "success", latencyMs: 450 }, ...]

Summary

• OmniRoute combos are database-stored routing policies that abstract multi-model provider selection.
• The combo routing engine in open-sse/services/combo.ts resolves targets, applies resilience filters, and iterates until success.
• Strategies include sequential (priority, weighted, auto) and parallel (fusion, pipeline) execution modes.
• Resilience mechanisms include circuit-breakers, cooldown windows, model lockouts, and quota-exhaustion thresholds.
• Full observability is provided via the X-OmniRoute-Combo-Trace header and diagnostic helpers.

Frequently Asked Questions

How does OmniRoute handle failover between model providers?

The combo engine implements cascading failover by iterating through the resolved target list in handleComboChatInner. If a provider returns an error or is filtered out by circuit-breaker or cooldown checks, the engine automatically proceeds to the next candidate until all targets are exhausted or maxSetRetries is reached.

What is the difference between the "auto" and "priority" combo strategies?

The priority strategy executes targets in the strict order defined in the combo configuration, while the auto strategy uses buildAutoCandidates to dynamically score and reorder targets based on real-time latency, cost, quota availability, and health status before execution begins.

How do I define and store a new combo in OmniRoute?

Define the combo as a JSON object specifying name, strategy, targets, and config, then persist it via the database layer in src/lib/db/combos.ts. The targets array must include provider identifiers, model strings, and optional connection-specific overrides.

Can I trace which specific model handled my request?

Yes. Every combo response includes the X-OmniRoute-Combo-Trace header, which contains a unique trace ID. Use the getComboTrace utility to retrieve the full decision log, including the specific model that succeeded, which targets were skipped due to circuit-breakers, and the latency of each attempt.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}