How to Create and Manage Model Combos in OmniRoute: A Complete Guide

Model combos in OmniRoute let you route a single LLM request to multiple providers using configurable strategies like priority, weighted, round-robin, auto, and fusion—complete with automatic fallback, rate-limiting, and quota management.

The combo system is OmniRoute's core abstraction for intelligent multi-provider request routing. Built on a modular, functional design, it separates pure configuration logic from stateful resilience mechanisms. This guide covers everything from defining a combo via JSON API to extending the pipeline with custom strategies.

---

Understanding the Combo Pipeline Architecture

The combo execution flow lives primarily in open-sse/services/combo.ts. The pipeline processes a request through eight distinct stages, each handled by dedicated modules:

Stage	Purpose	Source File
Configuration parsing	Validates combo name, models, strategy, and per-model settings	comboConfig.ts
Context setup	Creates ComboContext with request body, resolved config, and logging	combo/comboSetup.ts
Target resolution	Expands model lists into concrete ResolvedComboTargets with wildcard and fingerprint support	combo/comboStructure.ts
Auto-candidate generation	Ranks models by price, latency, quota, and circuit-breaker state	combo.ts (buildAutoCandidates)
Pre-dispatch filtering	Excludes targets in circuit-breaker open state, cooldown, or quota exhaustion	combo/comboPredicates.ts
Dispatch prelude	Handles pinned-model, fusion, chaos, and pipeline routing	combo/dispatchPrelude.ts
Execution loop	Retries with predictive TTFT circuit-breaker and quota-share concurrency	combo.ts (handleComboChat)
Diagnostics	Records attempt order, exhausted providers, and recovery hints	utils/error.ts

This separation enables unit testing of pure functions like resolveComboTargets and filterTargetsByRequestCompatibility while isolating stateful concerns (circuit-breakers, cooldown trackers) in dedicated utilities.

---

Defining a Model Combo via JSON API

Send combo definitions to the /v1/combo endpoint. The route delegates to handleComboChat, which calls resolveComboConfig in comboConfig.ts to parse the payload into a ComboLike object.

Weighted Strategy Example

{
  "name": "my-combo",
  "strategy": "weighted",
  "models": [
    { "model": "gpt-4o-mini", "weight": 3 },
    { "model": "claude-sonnet-4.6", "weight": 1 }
  ],
  "config": {
    "maxRetries": 2,
    "retryDelayMs": 1500,
    "predictiveTtftMs": 800,
    "fallbackCompressionMode": "gzip"
  }
}

Key fields:

• strategy: Determines routing algorithm (priority, weighted, round-robin, auto, fusion)
• models: Array of model references with strategy-specific parameters
• config.maxRetries: Per-target retry attempts before fallback
• config.predictiveTtftMs: Time-to-first-threshold for circuit-breaker decisions
• config.fallbackCompressionMode: Payload compression for fallback requests

---

Programmatic Combo Invocation

Use handleComboChat directly for server-side orchestration. This is the same function powering the HTTP route.

import { handleComboChat } from '@/open-sse/services/combo';
import { fetch } from 'node-fetch';

async function invokeCombo() {
  const body = {
    messages: [{ role: 'user', content: 'Explain quantum entanglement' }],
    max_tokens: 512,
  };

  const combo = {
    name: 'my-combo',
    strategy: 'weighted',
    models: [
      { model: 'gpt-4o-mini', weight: 3 },
      { model: 'claude-sonnet-4.6', weight: 1 },
    ],
    config: { maxRetries: 2, retryDelayMs: 1500 },
  };

  const response = await handleComboChat({
    body,
    combo,
    handleSingleModel: async (b, modelStr) => {
      // Execute against provider endpoint
      const res = await fetch(`https://api.provider.com/v1/${modelStr}`, {
        method: 'POST',
        body: JSON.stringify(b),
        headers: { 'Content-Type': 'application/json' },
      });
      return res;
    },
    log: console,
    settings: {},
    allCombos: [],
    relayOptions: {},
    signal: new AbortController().signal,
  });

  const result = await response.json();
  console.log(result);
}

The handleSingleModel callback is your integration point—swap the fetch implementation for custom authentication, caching, or telemetry.

---

Available Routing Strategies

Choose a strategy based on your reliability and cost requirements:

• priority: Try models in strict order, stopping at first success. Use for deterministic fallback chains.
• weighted: Distribute traffic by specified weights (e.g., 3:1 ratio in the example above). Good for cost optimization with backup.
• round-robin: Cycle evenly through models. Balances load but ignores performance differences.
• auto: Dynamically ranks candidates using buildAutoCandidates (lines 12-57 in combo.ts) based on real-time pricing, latency, quota, and circuit-breaker health. Best for autonomous optimization.
• fusion: Combines outputs from multiple models into a single response. Requires special handling in dispatchPrelude.ts.

---

Extending with Custom Strategies

Add new strategies by modifying resolveComboTargets in combo/comboStructure.ts. The function receives resolved targets and reorders them according to strategy logic.

Adding a Least-Latency Strategy

// open-sse/services/combo/comboStructure.ts
export function resolveComboTargets(
  combo: ComboLike,
  allCombos: ComboLike[],
  maxDepth: number,
  hiddenModelsByProvider: Map<string, Set<string>>
): ResolvedComboTarget[] {
  const targets = /* base resolution logic */;

  switch (combo.strategy) {
    case 'priority':
      return targets; // already ordered
    case 'weighted':
      return applyWeightedDistribution(targets, combo.models);
    case 'round-robin':
      return rotateFromLastUsed(targets);
    case 'least-latency':
      return targets.sort((a, b) => a.latencyMs - b.latencyMs);
    // ...
  }
}

After deployment, invoke with "strategy": "least-latency" in your combo definition.

---

Resilience and Fallback Mechanisms

OmniRoute's combo system implements multiple protective layers:

Provider-Level Guards

The comboPredicates.ts module evaluates targets before dispatch:

// Pseudo-code based on actual implementation
const shouldSkip = (
  isCircuitBreakerOpen(provider) ||
  isProviderInCooldown(provider) ||
  isModelLocked(model) ||
  isCredentialGateBlocked(credentials) ||
  exceedsQuotaExhaustionThreshold(usage)
);

Auto-Candidate Intelligence

The buildAutoCandidates function (lines 12-57 in combo.ts) constructs a ranked pool using:

• Pricing data: Per-token costs from provider rate cards
• Latency percentiles: Historical TTFT measurements from comboMetrics.ts
• Quota headroom: Remaining volume vs. soft limits
• Circuit-breaker state: Open/half-closed/closed per provider-model pair

Retry and Compression

The execution loop in handleComboChat (lines 63-150) applies:

• Exponential backoff with retryDelayMs jitter
• Predictive TTFT circuit-breaker: Aborts slow-starting responses before full timeout
• Quota-share concurrency: Caps parallel requests to preserve quota buffers
• Fallback compression: gzip or brotli reduction for retry payloads

---

Monitoring and Diagnostics

Track combo performance through built-in instrumentation:

Metric	Source	Purpose
Attempt order	error.ts	Debug routing decisions
Pool size	handleComboChat	Verify target expansion
Exhausted providers	error.ts	Identify systemic failures
Recovery hints	pinRecovery.ts	Suggest alternative combos

Metrics aggregation in comboMetrics.ts feeds back into auto-candidate scoring, creating a self-tuning routing system.

---

Key Source Files Reference

File	Responsibility
open-sse/services/combo.ts	Main orchestrator with handleComboChat and buildAutoCandidates
open-sse/services/comboConfig.ts	resolveComboConfig for payload validation
open-sse/services/combo/comboSetup.ts	ComboContext initialization
open-sse/services/combo/comboStructure.ts	Target resolution and strategy dispatch
open-sse/services/combo/comboPredicates.ts	Guard predicates for filtering
open-sse/services/combo/dispatchPrelude.ts	Special routing modes (fusion, chaos, pipeline)
open-sse/services/combo/comboMetrics.ts	Performance tracking and feedback

---

Summary

• Define combos via JSON to /v1/combo or programmatically through handleComboChat
• Choose strategies (priority, weighted, round-robin, auto, fusion) based on cost, reliability, and latency needs
• Extend the system by adding cases to resolveComboTargets in comboStructure.ts
• Leverage resilience through circuit-breakers, predictive TTFT, quota management, and automatic fallback
• Monitor execution via structured diagnostics and self-tuning metrics feedback

The modular architecture—pure functions for logic, isolated utilities for state—makes OmniRoute's combo system both production-hardened and extensible for custom use cases.

---

Frequently Asked Questions

How do I enable automatic provider selection based on performance?

Use the "auto" strategy in your combo definition. The buildAutoCandidates function in combo.ts automatically ranks providers using real-time pricing, latency percentiles from comboMetrics.ts, quota headroom, and circuit-breaker health. No manual configuration required—the system adapts to observed performance.

Can I combine outputs from multiple models into one response?

Yes. Set "strategy": "fusion" and ensure your request flows through dispatchPrelude.ts, which routes to the fusion dispatcher. This mode invokes multiple models concurrently and aggregates results before returning. Note that fusion requires appropriate handling in your handleSingleModel executor to manage parallel execution.

What happens when all providers in a combo fail?

The execution loop in handleComboChat (lines 63-150) exhausts all candidates, then generates a structured error via error.ts with attempt order, exhausted providers, and a recovery hint from pinRecovery.ts. You can configure maxRetries and retryDelayMs to control per-target retry behavior before final failure.

How do I add rate limiting to a specific model in a combo?

Apply per-model configuration in the models array of your combo definition. The comboConfig.ts parser supports provider-specific settings that comboPredicates.ts evaluates during pre-dispatch filtering. For global rate limits, configure src/lib/resilience/settings.ts with quota preflight and provider cooldown thresholds.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}