# How to Configure Multi-Provider Fallback Chains with OmniRoute Routing Strategies

> Configure multi provider fallback chains with OmniRoute. Ensure LLM resilience by setting up ordered provider sequences for automatic failover with declarative routing strategies.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-19

---

**OmniRoute enables resilient LLM request handling by allowing you to define ordered provider sequences called combos that automatically failover to secondary providers when primaries fail, controlled by declarative routing strategies.**

OmniRoute is an open-source LLM gateway that abstracts multiple providers behind a unified interface. By configuring multi-provider fallback chains, you ensure high availability for AI-powered applications even when individual providers experience rate limits or outages.

## Understanding Combo-Based Routing Architecture

OmniRoute implements failover logic through three core primitives: **combos**, **routing strategies**, and **resolvers**. Understanding these components is essential before implementing fallback chains.

### The Combo Abstraction

A **combo** is a named collection of provider-model pairs stored in the database layer at [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts). Each combo entry specifies a target provider, model identifier, and optional credentials. When a request arrives, OmniRoute expands this collection into executable targets using `resolveComboTargets()` before attempting execution.

Combos are validated against Zod schemas defined in [`src/shared/validation/schemas/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/schemas/combo.ts), which enforce that strategy names match the canonical list exported from [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts).

### Routing Strategies Overview

The `ROUTING_STRATEGY_VALUES` constant in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts) defines how OmniRoute orders and selects targets within a combo:

- **priority**: Attempts targets in the exact order defined in the configuration, creating a strict fallback chain.
- **fill-first**: Exhausts the first target's quota before moving to subsequent entries.
- **weighted**: Selects targets probabilistically based on assigned weight values, retrying with remaining candidates on failure.
- **least-used**: Routes to the provider with the lowest recent invocation count.
- **cost-optimized**: Automatically selects the cheapest provider that satisfies the token budget constraints.

## Defining Fallback Chains in Practice

Configuring a robust fallback chain requires creating a combo definition and attaching the appropriate strategy to handle failure scenarios.

### Creating a Combo Configuration

Define your provider sequence as a JSON object where each target includes the provider identifier and model name. The following example establishes a three-tier fallback chain:

```json
{
  "name": "production-fallback",
  "strategy": "priority",
  "targets": [
    { "provider": "openai", "model": "gpt-4o-mini" },
    { "provider": "anthropic", "model": "claude-3-sonnet-20240229" },
    { "provider": "groq", "model": "llama3-70b-8192" }
  ]
}

```

When using the **weighted** strategy, include a `weight` property to influence selection probability:

```json
{
  "name": "balanced-fallback",
  "strategy": "weighted",
  "targets": [
    { "provider": "openai", "model": "gpt-4o-mini", "weight": 5 },
    { "provider": "anthropic", "model": "claude-3-haiku-20240307", "weight": 3 },
    { "provider": "groq", "model": "mixtral-8x7b-32768", "weight": 2 }
  ]
}

```

### Selecting the Appropriate Strategy

Choose **priority** when you need deterministic failover behavior, ensuring requests only reach secondary providers when primaries return non-retryable errors. Use **weighted** for load distribution across healthy providers while maintaining implicit fallback capabilities. The **cost-optimized** strategy suits batch processing workloads where latency is less critical than expense management.

## Implementation Details from the Source Code

OmniRoute's fallback mechanism operates within the SSE (Server-Sent Events) service layer, specifically in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts).

### Resolution and Execution Flow

When handling a chat completion request, the system executes the following sequence:

1. The route handler extracts the combo name from the request body or user profile defaults.
2. `resolveComboTargets()` queries [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts) to retrieve the combo definition and expand it into a `ResolvedComboTarget[]` array.
3. `handleComboChat()` receives the resolved array and the selected strategy, then iterates over targets according to the strategy's ordering algorithm.
4. The Translator layer converts provider-specific response formats back to the client-facing schema.

### Fallback Trigger Mechanism

The resolver advances to the next target when it encounters specific error conditions. According to the implementation in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts), fallback triggers include HTTP 429 (rate limit), HTTP 500 (server error), and provider-specific "model unavailable" responses. This ensures that transient provider failures automatically activate your configured backup providers without client-side intervention.

## Configuration Examples

You can persist multi-provider fallback chains using either the command-line interface or the REST API endpoints located in `src/app/api/settings/combo/`.

### CLI-Based Setup

Create a JSON file containing your combo definition, then register it using the OmniRoute CLI:

```bash
cat > production-fallback.json <<'EOF'
{
  "name": "production-fallback",
  "strategy": "priority",
  "targets": [
    { "provider": "openai", "model": "gpt-4o-mini" },
    { "provider": "anthropic", "model": "claude-3-sonnet-20240229" },
    { "provider": "groq", "model": "llama3-70b-8192" }
  ]
}
EOF

omniroute combo set --file production-fallback.json

```

Once registered, any request specifying `"combo": "production-fallback"` will attempt OpenAI first, fall back to Anthropic on failure, and finally try Groq if both previous providers are unavailable.

### REST API Configuration

For programmatic management, use the Settings API endpoint. The following cURL command creates a weighted fallback chain:

```bash
curl -X POST https://your-omniroute.domain/api/settings/combo \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMNIRUTE_API_KEY" \
  -d '{
    "name": "api-weighted-fallback",
    "strategy": "weighted",
    "targets": [
      { "provider": "openai", "model": "gpt-4o-mini", "weight": 7 },
      { "provider": "anthropic", "model": "claude-3-haiku-20240307", "weight": 3 }
    ]
  }'

```

The API validates the payload against the Zod schemas in [`src/shared/validation/schemas/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/schemas/combo.ts), ensuring the strategy value exists in `ROUTING_STRATEGY_VALUES` before persistence.

## Monitoring and Optimization

OmniRoute exposes combo performance metrics through [`src/lib/db/comboMetrics.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/comboMetrics.ts), allowing you to identify providers that frequently trigger fallbacks. The `list_combo_metrics` MCP tool defined in [`open-sse/mcp-server/schemas/tools.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/mcp-server/schemas/tools.ts) queries success and failure counts per combo entry.

For cost-sensitive workloads, enable the **cost-optimized** strategy and update provider pricing data via [`src/lib/pricingSync.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/pricingSync.ts). This configuration automatically routes requests to the cheapest available provider while maintaining your fallback chain for reliability.

## Summary

- **Combos** are named provider collections stored in [`src/lib/db/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/combo.ts) that define your fallback candidate pool.
- **Routing strategies** (priority, weighted, cost-optimized, etc.) in [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts) determine failover order and selection logic.
- The **resolver** in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) executes `handleComboChat()` to iterate through targets when errors occur.
- Configure chains via CLI (`omniroute combo set`) or REST API (`POST /api/settings/combo`) using validated JSON schemas.
- Monitor fallback frequency using combo metrics to optimize provider selection and cost efficiency.

## Frequently Asked Questions

### What error codes trigger a fallback to the next provider in the chain?

According to [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts), OmniRoute advances to the next target when it encounters HTTP 429 (rate limit), HTTP 500 (server errors), or provider-specific "model unavailable" responses. Standard retry logic applies to transient network errors, but non-retryable provider errors immediately activate the fallback mechanism.

### How does the weighted strategy handle provider failures?

When using the **weighted** strategy, the resolver first selects a target based on the probability distribution defined by weight values. If that provider fails, the system removes it from the current iteration pool and reselects from remaining targets using the same weight ratios. This ensures failed providers are skipped while maintaining load distribution across healthy alternatives.

### Can I use different routing strategies for different types of requests?

Yes. OmniRoute stores combos as independent configurations in the database, and each request specifies its desired combo via the request body or user profile defaults. You can create separate combos with different strategies—such as a "priority" combo for real-time chat and a "cost-optimized" combo for batch processing—and route requests accordingly through the `src/app/api/settings/combo/*` endpoints.

### Where is the list of valid routing strategy values defined?

The canonical list of strategy identifiers is exported as `ROUTING_STRATEGY_VALUES` from [`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts). This array is referenced by the Zod validation schemas in [`src/shared/validation/schemas/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/validation/schemas/combo.ts) to ensure only supported strategies can be persisted through the API or CLI interfaces.