# How to Create and Manage Model Combos in OmniRoute: A Complete Guide

> Master OmniRoute model combos! Learn to route LLM requests to multiple providers with priority, weighted, round-robin, auto, and fusion strategies. Includes fallback, rate-limiting, and quota management.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Model combos in OmniRoute let you route a single LLM request to multiple providers using configurable strategies like priority, weighted, round-robin, auto, and fusion—complete with automatic fallback, rate-limiting, and quota management.**

The **combo system** is OmniRoute's core abstraction for intelligent multi-provider request routing. Built on a modular, functional design, it separates pure configuration logic from stateful resilience mechanisms. This guide covers everything from defining a combo via JSON API to extending the pipeline with custom strategies.

---

## Understanding the Combo Pipeline Architecture

The combo execution flow lives primarily in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts). The pipeline processes a request through **eight distinct stages**, each handled by dedicated modules:

| Stage | Purpose | Source File |
|-------|---------|-------------|
| Configuration parsing | Validates combo name, models, strategy, and per-model settings | [`comboConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboConfig.ts) |
| Context setup | Creates `ComboContext` with request body, resolved config, and logging | [`combo/comboSetup.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo/comboSetup.ts) |
| Target resolution | Expands model lists into concrete `ResolvedComboTarget`s with wildcard and fingerprint support | [`combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo/comboStructure.ts) |
| Auto-candidate generation | Ranks models by price, latency, quota, and circuit-breaker state | [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) (`buildAutoCandidates`) |
| Pre-dispatch filtering | Excludes targets in circuit-breaker open state, cooldown, or quota exhaustion | [`combo/comboPredicates.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo/comboPredicates.ts) |
| Dispatch prelude | Handles pinned-model, fusion, chaos, and pipeline routing | [`combo/dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo/dispatchPrelude.ts) |
| Execution loop | Retries with predictive TTFT circuit-breaker and quota-share concurrency | [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) (`handleComboChat`) |
| Diagnostics | Records attempt order, exhausted providers, and recovery hints | [`utils/error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/utils/error.ts) |

This separation enables **unit testing of pure functions** like `resolveComboTargets` and `filterTargetsByRequestCompatibility` while isolating stateful concerns (circuit-breakers, cooldown trackers) in dedicated utilities.

---

## Defining a Model Combo via JSON API

Send combo definitions to the `/v1/combo` endpoint. The route delegates to `handleComboChat`, which calls `resolveComboConfig` in [`comboConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboConfig.ts) to parse the payload into a `ComboLike` object.

### Weighted Strategy Example

```json
{
  "name": "my-combo",
  "strategy": "weighted",
  "models": [
    { "model": "gpt-4o-mini", "weight": 3 },
    { "model": "claude-sonnet-4.6", "weight": 1 }
  ],
  "config": {
    "maxRetries": 2,
    "retryDelayMs": 1500,
    "predictiveTtftMs": 800,
    "fallbackCompressionMode": "gzip"
  }
}

```

**Key fields:**
- `strategy`: Determines routing algorithm (`priority`, `weighted`, `round-robin`, `auto`, `fusion`)
- `models`: Array of model references with strategy-specific parameters
- `config.maxRetries`: Per-target retry attempts before fallback
- `config.predictiveTtftMs`: Time-to-first-threshold for circuit-breaker decisions
- `config.fallbackCompressionMode`: Payload compression for fallback requests

---

## Programmatic Combo Invocation

Use `handleComboChat` directly for server-side orchestration. This is the same function powering the HTTP route.

```typescript
import { handleComboChat } from '@/open-sse/services/combo';
import { fetch } from 'node-fetch';

async function invokeCombo() {
  const body = {
    messages: [{ role: 'user', content: 'Explain quantum entanglement' }],
    max_tokens: 512,
  };

  const combo = {
    name: 'my-combo',
    strategy: 'weighted',
    models: [
      { model: 'gpt-4o-mini', weight: 3 },
      { model: 'claude-sonnet-4.6', weight: 1 },
    ],
    config: { maxRetries: 2, retryDelayMs: 1500 },
  };

  const response = await handleComboChat({
    body,
    combo,
    handleSingleModel: async (b, modelStr) => {
      // Execute against provider endpoint
      const res = await fetch(`https://api.provider.com/v1/${modelStr}`, {
        method: 'POST',
        body: JSON.stringify(b),
        headers: { 'Content-Type': 'application/json' },
      });
      return res;
    },
    log: console,
    settings: {},
    allCombos: [],
    relayOptions: {},
    signal: new AbortController().signal,
  });

  const result = await response.json();
  console.log(result);
}

```

The `handleSingleModel` callback is your integration point—swap the fetch implementation for custom authentication, caching, or telemetry.

---

## Available Routing Strategies

Choose a strategy based on your reliability and cost requirements:

- **priority**: Try models in strict order, stopping at first success. Use for deterministic fallback chains.
- **weighted**: Distribute traffic by specified weights (e.g., 3:1 ratio in the example above). Good for cost optimization with backup.
- **round-robin**: Cycle evenly through models. Balances load but ignores performance differences.
- **auto**: Dynamically ranks candidates using `buildAutoCandidates` (lines 12-57 in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts)) based on real-time pricing, latency, quota, and circuit-breaker health. Best for autonomous optimization.
- **fusion**: Combines outputs from multiple models into a single response. Requires special handling in [`dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/dispatchPrelude.ts).

---

## Extending with Custom Strategies

Add new strategies by modifying `resolveComboTargets` in [`combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo/comboStructure.ts). The function receives resolved targets and reorders them according to strategy logic.

### Adding a Least-Latency Strategy

```typescript
// open-sse/services/combo/comboStructure.ts
export function resolveComboTargets(
  combo: ComboLike,
  allCombos: ComboLike[],
  maxDepth: number,
  hiddenModelsByProvider: Map<string, Set<string>>
): ResolvedComboTarget[] {
  const targets = /* base resolution logic */;

  switch (combo.strategy) {
    case 'priority':
      return targets; // already ordered
    case 'weighted':
      return applyWeightedDistribution(targets, combo.models);
    case 'round-robin':
      return rotateFromLastUsed(targets);
    case 'least-latency':
      return targets.sort((a, b) => a.latencyMs - b.latencyMs);
    // ...
  }
}

```

After deployment, invoke with `"strategy": "least-latency"` in your combo definition.

---

## Resilience and Fallback Mechanisms

OmniRoute's combo system implements **multiple protective layers**:

### Provider-Level Guards

The [`comboPredicates.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboPredicates.ts) module evaluates targets before dispatch:

```typescript
// Pseudo-code based on actual implementation
const shouldSkip = (
  isCircuitBreakerOpen(provider) ||
  isProviderInCooldown(provider) ||
  isModelLocked(model) ||
  isCredentialGateBlocked(credentials) ||
  exceedsQuotaExhaustionThreshold(usage)
);

```

### Auto-Candidate Intelligence

The `buildAutoCandidates` function (lines 12-57 in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts)) constructs a ranked pool using:

- **Pricing data**: Per-token costs from provider rate cards
- **Latency percentiles**: Historical TTFT measurements from [`comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboMetrics.ts)
- **Quota headroom**: Remaining volume vs. soft limits
- **Circuit-breaker state**: Open/half-closed/closed per provider-model pair

### Retry and Compression

The execution loop in `handleComboChat` (lines 63-150) applies:

- Exponential backoff with `retryDelayMs` jitter
- **Predictive TTFT circuit-breaker**: Aborts slow-starting responses before full timeout
- **Quota-share concurrency**: Caps parallel requests to preserve quota buffers
- **Fallback compression**: `gzip` or `brotli` reduction for retry payloads

---

## Monitoring and Diagnostics

Track combo performance through built-in instrumentation:

| Metric | Source | Purpose |
|--------|--------|---------|
| Attempt order | [`error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/error.ts) | Debug routing decisions |
| Pool size | `handleComboChat` | Verify target expansion |
| Exhausted providers | [`error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/error.ts) | Identify systemic failures |
| Recovery hints | [`pinRecovery.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/pinRecovery.ts) | Suggest alternative combos |

Metrics aggregation in [`comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboMetrics.ts) feeds back into auto-candidate scoring, creating a **self-tuning routing system**.

---

## Key Source Files Reference

| File | Responsibility |
|------|--------------|
| [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) | Main orchestrator with `handleComboChat` and `buildAutoCandidates` |
| [`open-sse/services/comboConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/comboConfig.ts) | `resolveComboConfig` for payload validation |
| [`open-sse/services/combo/comboSetup.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboSetup.ts) | `ComboContext` initialization |
| [`open-sse/services/combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboStructure.ts) | Target resolution and strategy dispatch |
| [`open-sse/services/combo/comboPredicates.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboPredicates.ts) | Guard predicates for filtering |
| [`open-sse/services/combo/dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/dispatchPrelude.ts) | Special routing modes (fusion, chaos, pipeline) |
| [`open-sse/services/combo/comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboMetrics.ts) | Performance tracking and feedback |

---

## Summary

- **Define combos** via JSON to `/v1/combo` or programmatically through `handleComboChat`
- **Choose strategies** (`priority`, `weighted`, `round-robin`, `auto`, `fusion`) based on cost, reliability, and latency needs
- **Extend the system** by adding cases to `resolveComboTargets` in [`comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboStructure.ts)
- **Leverage resilience** through circuit-breakers, predictive TTFT, quota management, and automatic fallback
- **Monitor execution** via structured diagnostics and self-tuning metrics feedback

The modular architecture—pure functions for logic, isolated utilities for state—makes OmniRoute's combo system both **production-hardened** and **extensible for custom use cases**.

---

## Frequently Asked Questions

### How do I enable automatic provider selection based on performance?

Use the `"auto"` strategy in your combo definition. The `buildAutoCandidates` function in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) automatically ranks providers using real-time pricing, latency percentiles from [`comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboMetrics.ts), quota headroom, and circuit-breaker health. No manual configuration required—the system adapts to observed performance.

### Can I combine outputs from multiple models into one response?

Yes. Set `"strategy": "fusion"` and ensure your request flows through [`dispatchPrelude.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/dispatchPrelude.ts), which routes to the fusion dispatcher. This mode invokes multiple models concurrently and aggregates results before returning. Note that fusion requires appropriate handling in your `handleSingleModel` executor to manage parallel execution.

### What happens when all providers in a combo fail?

The execution loop in `handleComboChat` (lines 63-150) exhausts all candidates, then generates a structured error via [`error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/error.ts) with attempt order, exhausted providers, and a recovery hint from [`pinRecovery.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/pinRecovery.ts). You can configure `maxRetries` and `retryDelayMs` to control per-target retry behavior before final failure.

### How do I add rate limiting to a specific model in a combo?

Apply per-model configuration in the `models` array of your combo definition. The [`comboConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboConfig.ts) parser supports provider-specific settings that [`comboPredicates.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboPredicates.ts) evaluates during pre-dispatch filtering. For global rate limits, configure [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) with quota preflight and provider cooldown thresholds.