# How OmniRoute Handles Routing: Inside the Combo Routing Engine

> Discover how OmniRoute handles routing with its Combo Routing Engine. Explore 17 strategies and find your optimal upstream provider for efficient, reliable connections.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-29

---

**OmniRoute handles routing through its Combo Routing Engine, which selects optimal upstream providers using 17 configurable strategies—including weighted, cost-optimized, and context-aware—by iterating through resolved targets in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) until a successful response is obtained or emergency fallback is triggered.**

OmniRoute is an open-source LLM gateway that optimizes request distribution across multiple AI providers. Understanding how OmniRoute handles routing is essential for operating large-scale AI workloads that demand high availability and cost efficiency. The system's routing logic centers on **combos**—configurable provider sequences that balance latency, cost, and reliability through dynamic strategy selection and automatic failover.

## The Combo-Centric Routing Architecture

OmniRoute’s routing optimization revolves around the **Combo Routing Engine**, which treats each request as a configurable workflow rather than a single endpoint call. This architecture separates routing into distinct phases: definition, resolution, strategy application, and execution.

### Combo Definition and Storage

A **combo** is a configuration object stored in the SQLite database ([`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts)) that lists one or more provider-model pairs. Each combo supports optional weighting, fallback rules, and priority settings. Combos can be created through the admin UI or CLI, making routing logic transparent and version-controlled.

### Target Resolution

When a request arrives, `open‑sse/services/combo.ts` invokes `resolveComboTargets()`. This function expands the combo definition into an ordered array of **ResolvedComboTarget** objects. Each target contains the concrete provider ID, model name, account credentials (resolved from encrypted DB fields via [`src/lib/db/encryption.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/encryption.ts)), and per-target overrides.

## The 17 Routing Strategies Explained

OmniRoute supports **17 routing strategies** defined in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts). The chosen strategy determines how resolved targets are traversed:

- **Priority**: Sequential failover through ordered targets.
- **Weighted**: Distribution based on configured percentages (e.g., 70% OpenAI, 30% Groq).
- **Round-robin**: Cyclical distribution across targets.
- **P2C (Power of Two Choices)**: Selects between two randomly chosen targets based on load.
- **Cost-optimized**: Selects the cheapest viable provider using live data from [`src/lib/pricingSync.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/pricingSync.ts).
- **Context-optimized**: Routes based on token consumption headroom and context window availability.

Other strategies include random selection, latency-optimized, and various hybrid approaches. The strategy identifier is attached to the combo configuration and evaluated during the execution phase.

## Execution Flow: From Request to Response

The actual routing execution follows a strict pipeline that ensures reliability and real-time optimization.

### The Combo Execution Loop

`handleComboChat()` in `open‑sse/services/combo.ts` implements the core execution loop. For each resolved target, it invokes `handleSingleModel()`, which wraps the standard chat handling pipeline from `open‑sse/handlers/chatCore.ts` with combo-specific logic. The loop includes circuit-breaker checks and error handling. If a target succeeds, the response returns immediately; otherwise, the loop proceeds to the next target according to the selected strategy until the list is exhausted.

### Real-Time Dynamic Adjustments

The routing engine reacts to runtime signals to optimize selection:

- **Rate-limit headers** from providers cause immediate deprioritization of overloaded targets.
- **Cost metrics** from [`src/lib/pricingSync.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/pricingSync.ts) enable the *cost-optimized* strategy to pick the cheapest viable provider.
- **Context-usage statistics** (e.g., token consumption) feed the *context-optimized* strategy, preferring providers with available headroom.

These adjustments happen during the execution loop without requiring configuration changes.

## Fallback and Resilience Mechanisms

If all primary targets fail, OmniRoute activates the **global emergency fallback** implemented in `open‑sse/services/emergencyFallback.ts`. This fallback can route to a self-hosted model or an inexpensive OpenAI-compatible endpoint, guaranteeing response availability during provider outages. This resilience layer ensures that routing failures never result in complete service unavailability.

## Configuring Routing in OmniRoute

OmniRoute exposes routing configuration through both CLI tools and API endpoints.

### Creating a Combo via CLI

Define weighted routing across multiple providers:

```bash
omniroute combo create \
  --name "fast‑and‑cheap" \
  --strategy weighted \
  --targets '[{"provider":"openai","model":"gpt-4o","weight":70},{"provider":"groq","model":"llama3-70b","weight":30}]'

```

This command persists the combo to the SQLite `combos` table and registers the weighted strategy.

### Using Combos in API Requests

Reference combos using the `combo:` prefix in model parameters:

```json
POST /api/v1/chat/completions
{
  "model": "combo:fast‑and‑cheap",
  "messages": [{ "role": "user", "content": "Explain quantum tunneling." }]
}

```

Inside `open‑sse/services/combo.ts`, the model string triggers `resolveComboTargets()` to fetch the configuration and `handleComboChat()` to execute the weighted selection logic.

### Monitoring Routing Performance

Access execution metrics programmatically:

```typescript
import { getComboMetrics } from "@omniroute/open-sse/mcp-tools";

const metrics = await getComboMetrics({ comboId: "fast‑and‑cheap" });
console.log(metrics); // { successRate: 0.97, avgLatencyMs: 210, costPerMTokens: 0.002 }

```

Metrics are aggregated from execution logs stored in [`src/lib/db/comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/comboMetrics.ts) and help operators tune strategies based on actual performance data.

## Summary

- OmniRoute handles routing through the **Combo Routing Engine**, which processes requests via configurable provider sequences stored in [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts).
- **17 routing strategies**—including weighted, cost-optimized, and context-aware—determine target traversal order, with identifiers defined in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts).
- The execution loop in `open‑sse/services/combo.ts` iterates through **ResolvedComboTarget** objects until success, with circuit-breaker logic and error handling.
- **Dynamic adjustments** respond to rate limits, pricing updates from [`src/lib/pricingSync.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/pricingSync.ts), and context usage in real-time.
- **Emergency fallback** via `open‑sse/services/emergencyFallback.ts` ensures availability when all primary targets fail.

## Frequently Asked Questions

### What is a combo in OmniRoute?

A combo is a routing configuration that defines one or more provider-model pairs with optional weights, priorities, and fallback rules. Combos are stored in the SQLite database via [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts) and serve as the foundational unit for OmniRoute's routing decisions, allowing administrators to treat multiple providers as a single logical endpoint.

### How does OmniRoute choose which provider to use?

OmniRoute selects providers based on the **routing strategy** attached to the combo configuration. The `resolveComboTargets()` function in `open‑sse/services/combo.ts` expands the combo into ordered targets, and `handleComboChat()` traverses them according to strategies like weighted distribution, cost-optimization using data from [`src/lib/pricingSync.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/pricingSync.ts), or priority ordering defined in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts).

### What happens if all providers in a combo fail?

If all targets fail, OmniRoute triggers the **global emergency fallback** located in `open‑sse/services/emergencyFallback.ts`. This routes the request to a self-hosted model or inexpensive backup endpoint, ensuring continuous availability even during widespread provider outages.

### How many routing strategies does OmniRoute support?

OmniRoute supports **17 distinct routing strategies**, including priority-based, weighted, round-robin, P2C (Power of Two Choices), cost-optimized, and context-optimized approaches. The complete list of strategy identifiers is maintained in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts).