# What Is a "Combo" in OmniRoute and How Does It Work?

> Discover what a combo is in OmniRoute and how this model-routing construct dispatches requests to multiple LLM targets with configurable strategies, automatic fallback, and load balancing.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-31

---

**A combo in OmniRoute is a model-routing construct that dispatches a single request to multiple LLM targets using configurable strategies, automatic fallback, and load-balancing logic.**

A combo (short for "model combo") serves as the core abstraction in [diegosouzapw/OmniRoute](https://github.com/diegosouzapw/OmniRoute) for building resilient, cost-aware AI request flows. Instead of hardcoding a single provider or model, you define a **strategy**, a list of **candidate models**, and optional **configuration** (timeouts, retries, quotas). The runtime evaluates candidates, selects the best-fit target, and transparently falls back to the next one on failure.

---

## Combo Structure and Configuration

Every combo payload contains four essential fields:

| Field | Purpose | Example |
|-------|---------|---------|
| `name` | Identifier for logging and metrics | `"my-priority-combo"` |
| `strategy` | Selection algorithm (19 built-in options) | `"priority"`, `"auto"`, `"fusion"` |
| `models` | Ordered list of target candidates | `[{ "model": "claude-opus-4.6", "connectionId": "conn-123" }]` |
| `config` | Resilience parameters | `{ "maxRetries": 2, "comboTimeoutMs": 8000 }` |

### Example Combo Payload

```json
{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Explain combo routing." }],
  "combo": {
    "name": "my-priority-combo",
    "strategy": "priority",
    "models": [
      { "model": "claude-opus-4.6", "connectionId": "conn-123" },
      { "model": "gpt-4o-mini", "connectionId": "conn-456" }
    ],
    "config": {
      "maxRetries": 2,
      "retryDelayMs": 1500,
      "comboTimeoutMs": 8000
    }
  }
}

```

This payload is sent to `POST /v1/chat/completions`, implemented as a Next.js API route under `src/app/api/v1/chat/completions`.

---

## How Combo Routing Executes: 10-Stage Pipeline

The combo execution flow in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) follows a rigorous pipeline from request ingestion to response delivery.

### Stage 1: Context Initialization

`handleComboChat` (lines 33–45) creates a `ComboContext` that bundles the request body, combo metadata, and a per-request logger. According to the OmniRoute source code, this context object travels through all subsequent stages, accumulating diagnostic data.

### Stage 2: Dispatch Prelude (Strategy Branching)

Before the generic target loop, [`dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/dispatchPrelude.ts) handles special strategies:

- **Pinned-model dispatch** — Skips normal selection for sticky sessions
- **Fusion** — Parallel fan-out to multiple models with a judge model aggregation
- **Chaos** — Parallel multi-model execution for A/B testing

These branches live in [`open-sse/services/combo/dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/dispatchPrelude.ts) and short-circuit the standard flow when activated.

### Stage 3: Target Resolution

`resolveComboTargets` in [`comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboStructure.ts) (lines 53–66) expands raw model references into concrete `ResolvedComboTarget` objects. This phase applies:

- Hidden-model filters
- Fingerprint expansion for connection variants
- Provider-wildcard handling

### Stage 4: Candidate Scoring (Auto-Combo)

For the `auto` strategy, `buildAutoCandidates` (lines 16–58) gathers real-time signals:

- **Latency** — Historical response times per model
- **Cost** — Price per 1M tokens
- **Quota status** — Remaining budget and reset windows
- **Circuit-breaker state** — Whether the provider is healthy
- **Quality scores** — Success-rate metrics from `qualityScoreFor`

These factors produce a ranked candidate pool for optimal selection.

### Stage 5: Per-Target Validation Loop

`executeTarget` inside `dispatchWithCooldownRetry` (lines 24–53) validates each candidate before invocation:

```typescript
// Checks performed in order:
1. Provider circuit breaker (getCircuitBreaker)
2. Provider-wide cooldown (isProviderInCooldown)
3. Model lockout (isModelLocked)
4. Quota & reset-window status

```

Failed checks trigger **target skipping** with diagnostic recording, not request failure.

### Stage 6: Request Execution

The selected target invokes `handleSingleModelWithTimeout` (lines 86–89), which wraps the provider executor with `resolveTargetTimeoutMsForTarget` for per-target deadline enforcement.

### Stage 7: Fallback and Retry

Error responses activate the fallback loop. Global settings (`maxRetries`, `maxSetRetries`) and **cooldown-aware retries** via `waitForCooldownAwareRetry` (lines 99–108) may pause execution before retrying.

### Stage 8: Diagnostics and Tracing

Throughout execution, `startComboTrace` and `recordComboDecision` build a per-invocation trace attached to the response header `X-OmniRoute-Combo-Trace`. Error responses include a `ComboDiagnostics` payload listing pool size, attempted targets, and recovery hints.

### Stage 9: Metrics Recording

`recordComboRequest`, `recordComboShadowRequest`, and `getComboMetrics` in [`comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboMetrics.ts) capture operational data for observability.

### Stage 10: Cleanup

Stale pins (LKGP — "last known good provider", session stickiness) clear automatically after failures to prevent routing degradation.

---

## Built-In Combo Strategies

OmniRoute ships with 19 configurable strategies for different operational needs:

| Strategy | Use Case | Behavior |
|----------|----------|----------|
| **priority** | Cost control | Strict top-to-bottom ordering |
| **weighted** | Traffic shaping | Probabilistic selection by weight |
| **round-robin** | Load distribution | Cyclical rotation across candidates |
| **auto** | Dynamic optimization | Real-time scoring from [`autoStrategy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/autoStrategy.ts) |
| **fusion** | Quality maximization | Parallel execution + judge model |
| **chaos** | Testing/validation | Parallel multi-model for comparison |

Strategy selection happens at the dispatch prelude; custom strategies integrate by registering in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) (around lines 4–5).

---

## Resilience Mechanisms in Combo Routing

OmniRoute implements four layered defenses against provider failures:

**Circuit Breaker** ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) — Provider-level breaker with `OPEN`, `CLOSED`, and `HALF_OPEN` states. In `OPEN` state, targets skip immediately without network calls.

**Provider Cooldown** ([`providerCooldownTracker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/providerCooldownTracker.ts)) — Global rate-limit protection that blocks traffic for a configurable window after HTTP 429 or provider errors.

**Model Lockout** — Per-model quota or permission failures disable only the offending model while preserving connection viability for other models on the same provider.

**Quota & Reset-Windows** — [`quotaPreflight.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/quotaPreflight.ts) enforces per-connection token budgets and time-based quota resets, preventing cost overruns.

---

## Programmatic Combo Invocation

For testing or custom integrations, invoke the combo handler directly:

```typescript
import { handleComboChat } from '@/open-sse/services/combo';
import { handleChatCore } from '@/open-sse/handlers/chatCore';

await handleComboChat({
  body: requestBody,
  combo: requestBody.combo,
  handleSingleModel: async (b, model) => {
    // Forward to standard chat handler
    return handleChatCore({ body: b, modelStr: model });
  },
  isModelAvailable: undefined,
  log: console,
  settings: {},      // Default resilience
  allCombos: [],     // Other combo definitions
  signal: undefined,
});

```

This pattern enables unit testing of combo logic without HTTP overhead.

---

## Adding Custom Combo Strategies

Extend OmniRoute with domain-specific routing logic:

```typescript
// open-sse/services/combo/strategies/costOptimized.ts
export async function tryCostOptimized(options: StrategyOptions) {
  // Sort by cost per 1M tokens, lowest first
  const ordered = options.candidates.sort(
    (a, b) => a.costPer1MTokens - b.costPer1MTokens
  );
  return dispatchWithOrderedTargets(ordered, options);
}

```

Register in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) and reference by name in combo payloads.

---

## Summary

- A **combo** is OmniRoute's declarative routing primitive for multi-model, resilient LLM requests
- **19 built-in strategies** cover priority, weighted, round-robin, auto-scored, and parallel execution patterns
- **Ten-stage pipeline** from `handleComboChat` through diagnostics ensures transparent fallback and observability
- **Four resilience layers** — circuit breaker, provider cooldown, model lockout, quota enforcement — protect against cascading failures
- **Extensible architecture** allows custom strategies via the `StrategyOptions` interface

---

## Frequently Asked Questions

### What is the difference between a combo and a single model request?

A single model request targets one provider-model pair directly. A combo wraps that request with candidate lists, strategy-based selection, and automatic fallback. The `combo` field in the payload activates the routing pipeline; omitting it bypasses all combo logic.

### How does the "auto" strategy choose between models?

The auto strategy calls `buildAutoCandidates` to score each candidate across latency, cost, quota availability, circuit-breaker health, and historical success rates. The highest-scoring candidate receives the request; if it fails, the runner-up attempts next.

### Can I use combos with streaming responses?

Yes. The combo pipeline in `handleComboChat` supports streaming via the same `handleSingleModelWithTimeout` abstraction. The trace header and diagnostics attach to the final response metadata even for streaming sessions.

### Where are combo metrics and traces stored?

Per-request traces populate the `X-OmniRoute-Combo-Trace` response header and `ComboDiagnostics` error payloads. Aggregated metrics flow through `recordComboRequest` and `recordComboShadowRequest` in [`comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboMetrics.ts), ready for external observability platform integration.