# How to Configure Provider Fallback Chains in OMP for Automatic Model Failover

> Configure provider fallback chains in OMP for automatic model failover. Set allow fallbacks true and define an order array in your model config for seamless failover when the primary provider fails.

- Repository: [Can Bölük/oh-my-pi](https://github.com/can1357/oh-my-pi)
- Tags: how-to-guide
- Published: 2026-05-21

---

**OMP (Oh My Pi) enables automatic model failover by defining an `order` array of alternative providers and setting `allow_fallbacks: true` in your model configuration, which `AgentSession` traverses transparently when the primary provider fails.**

OMP (Oh My Pi) implements resilient AI model access through configurable provider fallback chains that automatically reroute requests when the primary endpoint encounters rate limits, network errors, or credential issues. To configure provider fallback chains in OMP, you define an ordered list of alternative providers in [`opencode.json`](https://github.com/can1357/oh-my-pi/blob/main/opencode.json) or the generated [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml), coupled with the `allow_fallbacks` boolean flag that controls whether the system should attempt automatic failover. This mechanism ensures high availability without manual intervention when a specific provider becomes unreachable.

## Understanding Provider Fallback Mechanisms

OMP's model registry reads configuration from [`opencode.json`](https://github.com/can1357/oh-my-pi/blob/main/opencode.json) (and the generated [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml)) to build the runtime provider resolution logic. For each model, you can declare a fallback chain that specifies exactly which alternative providers to attempt if the primary request fails.

The fallback system evaluates two critical fields within the `provider` configuration block:

- **`order`** – An array of provider identifiers that defines the exact sequence to attempt during failover.
- **`allow_fallbacks`** – A boolean flag that enables (`true`) or disables (`false`) the automatic retry logic.

When a request errors, `AgentSession` first checks `contextPromotion.enabled` to handle context-size overflow scenarios. If no promotion target exists, the system evaluates the provider fallback chain. This chain operates **role-agnostically**, meaning it simply selects the next provider in the `order` array that possesses a valid API key according to `ModelRegistry.getApiKey()`. The switch occurs transparently, and the retry reissues the request with the new provider credentials.

## Configuring Fallback Chains in opencode.json

The repository-wide [`opencode.json`](https://github.com/can1357/oh-my-pi/blob/main/opencode.json) file serves as the primary configuration source for the Instagit UI and the model registry. Define your fallback sequence under the model-specific `options.provider` block.

```json
{
  "provider": {
    "openrouter": {
      "options": {
        "headers": {
          "OpenRouter-App-Title": "Instagit",
          "OpenRouter-App-URL": "https://instagit.com"
        }
      },
      "models": {
        "openai/gpt-oss-120b": {
          "options": {
            "provider": {
              "order": ["fireworks", "cerebras"],
              "allow_fallbacks": true
            }
          }
        },
        "qwen/qwen3-32b": {
          "options": {
            "provider": {
              "order": ["sambanova", "groq"],
              "allow_fallbacks": true
            }
          }
        }
      }
    }
  }
}

```

In this example, when `openai/gpt-oss-120b` fails on the `openrouter` provider, OMP automatically attempts the request through `fireworks`, then `cerebras` if the first alternative also fails.

## Defining Fallback Sequences in models.yml

The generated [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml) file supports the same fallback configuration using YAML syntax. This file is consumed by [`packages/coding-agent/src/config/model-registry.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/coding-agent/src/config/model-registry.ts) to validate provider blocks and load API credentials.

```yaml
providers:
  openrouter:
    modelOverrides:
      openai/gpt-oss-120b:
        provider:
          order: [fireworks, cerebras]
          allow_fallbacks: true
      qwen/qwen3-32b:
        provider:
          order: [sambanova, groq]
          allow_fallbacks: true

```

Ensure each provider listed in the `order` array has a corresponding API key configured in `~/.omp/agent/models.container.yml` or your designated credential store.

## Runtime Failover Execution

The fallback logic resides in [`packages/ai/src/model-thinking.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/model-thinking.ts), which integrates deeply with `AgentSession` retry handling. When the primary provider fails, the system:

1. Validates that `allow_fallbacks` is enabled for the current model.
2. Iterates through the `order` array to find a provider with valid credentials via `ModelRegistry.getApiKey()`.
3. Rebuilds the request through [`packages/ai/src/providers/openrouter.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/providers/openrouter.ts) (or [`packages/ai/src/providers/openai-completions.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/providers/openai-completions.ts) for OpenAI-compatible endpoints).
4. Reissues the request transparently without returning an error to the caller.

This process requires no additional CLI flags or manual intervention. For example, if you trigger a request using the primary provider:

```bash
omp --model openai/gpt-oss-120b --provider openrouter

```

And the request encounters a rate limit, OMP automatically retries with `fireworks`, then `cerebras`, provided `allow_fallbacks` remains `true` and API keys exist for those providers.

## Key Source Files and Responsibilities

Understanding these implementation details helps troubleshoot failover behavior:

- **[`opencode.json`](https://github.com/can1357/oh-my-pi/blob/main/opencode.json)** – Contains the top-level provider configuration including `order` and `allow_fallbacks` declarations.
- **[`docs/models.md`](https://github.com/can1357/oh-my-pi/blob/main/docs/models.md)** – Documents the model-level fallback semantics and [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml) syntax according to the repository documentation.
- **[`packages/ai/src/providers/openrouter.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/providers/openrouter.ts)** – Implements the router that merges `order` and `allow_fallbacks` into the request-building logic.
- **[`packages/ai/src/model-thinking.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/model-thinking.ts)** – Applies the fallback chain during retry handling and coordinates with `AgentSession`.
- **[`packages/coding-agent/src/config/model-registry.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/coding-agent/src/config/model-registry.ts)** – Loads [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml) and validates provider blocks, managing API key retrieval through `getApiKey()`.

## Summary

To configure provider fallback chains in OMP for automatic model failover:

- Declare an **`order`** array in your model's `provider` configuration to specify the exact sequence of backup providers.
- Set **`allow_fallbacks: true`** to enable automatic traversal of the fallback chain during request failures.
- Ensure every provider in the chain has valid API credentials stored in your OMP configuration files.
- Restart long-running OMP sessions to reload configuration changes from [`opencode.json`](https://github.com/can1357/oh-my-pi/blob/main/opencode.json) or [`models.yml`](https://github.com/can1357/oh-my-pi/blob/main/models.yml).

## Frequently Asked Questions

### What triggers a provider fallback in OMP?

A fallback triggers when the primary provider returns an error such as a rate limit, network timeout, or invalid credential response. `AgentSession` catches these failures and, provided `allow_fallbacks` is `true`, initiates the next provider attempt from the configured `order` array without exposing the intermediate failure to the user.

### How does OMP validate providers before attempting a fallback?

Before switching to an alternative provider, OMP verifies the presence of a valid API key using `ModelRegistry.getApiKey(...)`. If the next provider in the `order` lacks credentials, the system skips to the subsequent entry until it finds a valid configuration or exhausts the chain.

### Can I disable fallback chains for specific models?

Yes. Set `allow_fallbacks: false` (the default for many models) in the model's `provider` configuration block. When disabled, `AgentSession` terminates the request immediately upon primary provider failure rather than attempting the fallback chain.

### Where does the retry logic reside in the OMP source code?

The retry logic is implemented in [`packages/ai/src/model-thinking.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/model-thinking.ts), which coordinates with `AgentSession` to evaluate the fallback chain. The actual request rebuilding and provider switching occur in [`packages/ai/src/providers/openrouter.ts`](https://github.com/can1357/oh-my-pi/blob/main/packages/ai/src/providers/openrouter.ts), which respects the `order` and `allow_fallbacks` fields when constructing new requests after a failure.