What Is a "Combo" in OmniRoute and How Does It Work?

A combo in OmniRoute is a model-routing construct that dispatches a single request to multiple LLM targets using configurable strategies, automatic fallback, and load-balancing logic.

A combo (short for "model combo") serves as the core abstraction in diegosouzapw/OmniRoute for building resilient, cost-aware AI request flows. Instead of hardcoding a single provider or model, you define a strategy, a list of candidate models, and optional configuration (timeouts, retries, quotas). The runtime evaluates candidates, selects the best-fit target, and transparently falls back to the next one on failure.

---

Combo Structure and Configuration

Every combo payload contains four essential fields:

Field	Purpose	Example
name	Identifier for logging and metrics	"my-priority-combo"
strategy	Selection algorithm (19 built-in options)	"priority", "auto", "fusion"
models	Ordered list of target candidates	[{ "model": "claude-opus-4.6", "connectionId": "conn-123" }]
config	Resilience parameters	{ "maxRetries": 2, "comboTimeoutMs": 8000 }

Example Combo Payload

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Explain combo routing." }],
  "combo": {
    "name": "my-priority-combo",
    "strategy": "priority",
    "models": [
      { "model": "claude-opus-4.6", "connectionId": "conn-123" },
      { "model": "gpt-4o-mini", "connectionId": "conn-456" }
    ],
    "config": {
      "maxRetries": 2,
      "retryDelayMs": 1500,
      "comboTimeoutMs": 8000
    }
  }
}

This payload is sent to POST /v1/chat/completions, implemented as a Next.js API route under src/app/api/v1/chat/completions.

---

How Combo Routing Executes: 10-Stage Pipeline

The combo execution flow in open-sse/services/combo.ts follows a rigorous pipeline from request ingestion to response delivery.

Stage 1: Context Initialization

handleComboChat (lines 33–45) creates a ComboContext that bundles the request body, combo metadata, and a per-request logger. According to the OmniRoute source code, this context object travels through all subsequent stages, accumulating diagnostic data.

Stage 2: Dispatch Prelude (Strategy Branching)

Before the generic target loop, dispatchPrelude.ts handles special strategies:

• Pinned-model dispatch — Skips normal selection for sticky sessions
• Fusion — Parallel fan-out to multiple models with a judge model aggregation
• Chaos — Parallel multi-model execution for A/B testing

These branches live in open-sse/services/combo/dispatchPrelude.ts and short-circuit the standard flow when activated.

Stage 3: Target Resolution

resolveComboTargets in comboStructure.ts (lines 53–66) expands raw model references into concrete ResolvedComboTarget objects. This phase applies:

• Hidden-model filters
• Fingerprint expansion for connection variants
• Provider-wildcard handling

Stage 4: Candidate Scoring (Auto-Combo)

For the auto strategy, buildAutoCandidates (lines 16–58) gathers real-time signals:

• Latency — Historical response times per model
• Cost — Price per 1M tokens
• Quota status — Remaining budget and reset windows
• Circuit-breaker state — Whether the provider is healthy
• Quality scores — Success-rate metrics from qualityScoreFor

These factors produce a ranked candidate pool for optimal selection.

Stage 5: Per-Target Validation Loop

executeTarget inside dispatchWithCooldownRetry (lines 24–53) validates each candidate before invocation:

// Checks performed in order:
1. Provider circuit breaker (getCircuitBreaker)
2. Provider-wide cooldown (isProviderInCooldown)
3. Model lockout (isModelLocked)
4. Quota & reset-window status

Failed checks trigger target skipping with diagnostic recording, not request failure.

Stage 6: Request Execution

The selected target invokes handleSingleModelWithTimeout (lines 86–89), which wraps the provider executor with resolveTargetTimeoutMsForTarget for per-target deadline enforcement.

Stage 7: Fallback and Retry

Error responses activate the fallback loop. Global settings (maxRetries, maxSetRetries) and cooldown-aware retries via waitForCooldownAwareRetry (lines 99–108) may pause execution before retrying.

Stage 8: Diagnostics and Tracing

Throughout execution, startComboTrace and recordComboDecision build a per-invocation trace attached to the response header X-OmniRoute-Combo-Trace. Error responses include a ComboDiagnostics payload listing pool size, attempted targets, and recovery hints.

Stage 9: Metrics Recording

recordComboRequest, recordComboShadowRequest, and getComboMetrics in comboMetrics.ts capture operational data for observability.

Stage 10: Cleanup

Stale pins (LKGP — "last known good provider", session stickiness) clear automatically after failures to prevent routing degradation.

---

Built-In Combo Strategies

OmniRoute ships with 19 configurable strategies for different operational needs:

Strategy	Use Case	Behavior
priority	Cost control	Strict top-to-bottom ordering
weighted	Traffic shaping	Probabilistic selection by weight
round-robin	Load distribution	Cyclical rotation across candidates
auto	Dynamic optimization	Real-time scoring from autoStrategy.ts
fusion	Quality maximization	Parallel execution + judge model
chaos	Testing/validation	Parallel multi-model for comparison

Strategy selection happens at the dispatch prelude; custom strategies integrate by registering in combo.ts (around lines 4–5).

---

Resilience Mechanisms in Combo Routing

OmniRoute implements four layered defenses against provider failures:

Circuit Breaker (src/shared/utils/circuitBreaker.ts) — Provider-level breaker with OPEN, CLOSED, and HALF_OPEN states. In OPEN state, targets skip immediately without network calls.

Provider Cooldown (providerCooldownTracker.ts) — Global rate-limit protection that blocks traffic for a configurable window after HTTP 429 or provider errors.

Model Lockout — Per-model quota or permission failures disable only the offending model while preserving connection viability for other models on the same provider.

Quota & Reset-Windows — quotaPreflight.ts enforces per-connection token budgets and time-based quota resets, preventing cost overruns.

---

Programmatic Combo Invocation

For testing or custom integrations, invoke the combo handler directly:

import { handleComboChat } from '@/open-sse/services/combo';
import { handleChatCore } from '@/open-sse/handlers/chatCore';

await handleComboChat({
  body: requestBody,
  combo: requestBody.combo,
  handleSingleModel: async (b, model) => {
    // Forward to standard chat handler
    return handleChatCore({ body: b, modelStr: model });
  },
  isModelAvailable: undefined,
  log: console,
  settings: {},      // Default resilience
  allCombos: [],     // Other combo definitions
  signal: undefined,
});

This pattern enables unit testing of combo logic without HTTP overhead.

---

Adding Custom Combo Strategies

Extend OmniRoute with domain-specific routing logic:

// open-sse/services/combo/strategies/costOptimized.ts
export async function tryCostOptimized(options: StrategyOptions) {
  // Sort by cost per 1M tokens, lowest first
  const ordered = options.candidates.sort(
    (a, b) => a.costPer1MTokens - b.costPer1MTokens
  );
  return dispatchWithOrderedTargets(ordered, options);
}

Register in combo.ts and reference by name in combo payloads.

---

Summary

• A combo is OmniRoute's declarative routing primitive for multi-model, resilient LLM requests
• 19 built-in strategies cover priority, weighted, round-robin, auto-scored, and parallel execution patterns
• Ten-stage pipeline from handleComboChat through diagnostics ensures transparent fallback and observability
• Four resilience layers — circuit breaker, provider cooldown, model lockout, quota enforcement — protect against cascading failures
• Extensible architecture allows custom strategies via the StrategyOptions interface

---

Frequently Asked Questions

What is the difference between a combo and a single model request?

A single model request targets one provider-model pair directly. A combo wraps that request with candidate lists, strategy-based selection, and automatic fallback. The combo field in the payload activates the routing pipeline; omitting it bypasses all combo logic.

How does the "auto" strategy choose between models?

The auto strategy calls buildAutoCandidates to score each candidate across latency, cost, quota availability, circuit-breaker health, and historical success rates. The highest-scoring candidate receives the request; if it fails, the runner-up attempts next.

Can I use combos with streaming responses?

Yes. The combo pipeline in handleComboChat supports streaming via the same handleSingleModelWithTimeout abstraction. The trace header and diagnostics attach to the final response metadata even for streaming sessions.

Where are combo metrics and traces stored?

Per-request traces populate the X-OmniRoute-Combo-Trace response header and ComboDiagnostics error payloads. Aggregated metrics flow through recordComboRequest and recordComboShadowRequest in comboMetrics.ts, ready for external observability platform integration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}