How to Configure Provider Fallback Chains in OMP for Automatic Model Failover

OMP (Oh My Pi) enables automatic model failover by defining an order array of alternative providers and setting allow_fallbacks: true in your model configuration, which AgentSession traverses transparently when the primary provider fails.

OMP (Oh My Pi) implements resilient AI model access through configurable provider fallback chains that automatically reroute requests when the primary endpoint encounters rate limits, network errors, or credential issues. To configure provider fallback chains in OMP, you define an ordered list of alternative providers in opencode.json or the generated models.yml, coupled with the allow_fallbacks boolean flag that controls whether the system should attempt automatic failover. This mechanism ensures high availability without manual intervention when a specific provider becomes unreachable.

Understanding Provider Fallback Mechanisms

OMP's model registry reads configuration from opencode.json (and the generated models.yml) to build the runtime provider resolution logic. For each model, you can declare a fallback chain that specifies exactly which alternative providers to attempt if the primary request fails.

The fallback system evaluates two critical fields within the provider configuration block:

• order – An array of provider identifiers that defines the exact sequence to attempt during failover.
• allow_fallbacks – A boolean flag that enables (true) or disables (false) the automatic retry logic.

When a request errors, AgentSession first checks contextPromotion.enabled to handle context-size overflow scenarios. If no promotion target exists, the system evaluates the provider fallback chain. This chain operates role-agnostically, meaning it simply selects the next provider in the order array that possesses a valid API key according to ModelRegistry.getApiKey(). The switch occurs transparently, and the retry reissues the request with the new provider credentials.

Configuring Fallback Chains in opencode.json

The repository-wide opencode.json file serves as the primary configuration source for the Instagit UI and the model registry. Define your fallback sequence under the model-specific options.provider block.

{
  "provider": {
    "openrouter": {
      "options": {
        "headers": {
          "OpenRouter-App-Title": "Instagit",
          "OpenRouter-App-URL": "https://instagit.com"
        }
      },
      "models": {
        "openai/gpt-oss-120b": {
          "options": {
            "provider": {
              "order": ["fireworks", "cerebras"],
              "allow_fallbacks": true
            }
          }
        },
        "qwen/qwen3-32b": {
          "options": {
            "provider": {
              "order": ["sambanova", "groq"],
              "allow_fallbacks": true
            }
          }
        }
      }
    }
  }
}

In this example, when openai/gpt-oss-120b fails on the openrouter provider, OMP automatically attempts the request through fireworks, then cerebras if the first alternative also fails.

Defining Fallback Sequences in models.yml

The generated models.yml file supports the same fallback configuration using YAML syntax. This file is consumed by packages/coding-agent/src/config/model-registry.ts to validate provider blocks and load API credentials.

providers:
  openrouter:
    modelOverrides:
      openai/gpt-oss-120b:
        provider:
          order: [fireworks, cerebras]
          allow_fallbacks: true
      qwen/qwen3-32b:
        provider:
          order: [sambanova, groq]
          allow_fallbacks: true

Ensure each provider listed in the order array has a corresponding API key configured in ~/.omp/agent/models.container.yml or your designated credential store.

Runtime Failover Execution

The fallback logic resides in packages/ai/src/model-thinking.ts, which integrates deeply with AgentSession retry handling. When the primary provider fails, the system:

1. Validates that allow_fallbacks is enabled for the current model.
2. Iterates through the order array to find a provider with valid credentials via ModelRegistry.getApiKey().
3. Rebuilds the request through packages/ai/src/providers/openrouter.ts (or packages/ai/src/providers/openai-completions.ts for OpenAI-compatible endpoints).
4. Reissues the request transparently without returning an error to the caller.

This process requires no additional CLI flags or manual intervention. For example, if you trigger a request using the primary provider:

omp --model openai/gpt-oss-120b --provider openrouter

And the request encounters a rate limit, OMP automatically retries with fireworks, then cerebras, provided allow_fallbacks remains true and API keys exist for those providers.

Key Source Files and Responsibilities

Understanding these implementation details helps troubleshoot failover behavior:

• opencode.json – Contains the top-level provider configuration including order and allow_fallbacks declarations.
• docs/models.md – Documents the model-level fallback semantics and models.yml syntax according to the repository documentation.
• packages/ai/src/providers/openrouter.ts – Implements the router that merges order and allow_fallbacks into the request-building logic.
• packages/ai/src/model-thinking.ts – Applies the fallback chain during retry handling and coordinates with AgentSession.
• packages/coding-agent/src/config/model-registry.ts – Loads models.yml and validates provider blocks, managing API key retrieval through getApiKey().

Summary

To configure provider fallback chains in OMP for automatic model failover:

• Declare an order array in your model's provider configuration to specify the exact sequence of backup providers.
• Set allow_fallbacks: true to enable automatic traversal of the fallback chain during request failures.
• Ensure every provider in the chain has valid API credentials stored in your OMP configuration files.
• Restart long-running OMP sessions to reload configuration changes from opencode.json or models.yml.

Frequently Asked Questions

What triggers a provider fallback in OMP?

A fallback triggers when the primary provider returns an error such as a rate limit, network timeout, or invalid credential response. AgentSession catches these failures and, provided allow_fallbacks is true, initiates the next provider attempt from the configured order array without exposing the intermediate failure to the user.

How does OMP validate providers before attempting a fallback?

Before switching to an alternative provider, OMP verifies the presence of a valid API key using ModelRegistry.getApiKey(...). If the next provider in the order lacks credentials, the system skips to the subsequent entry until it finds a valid configuration or exhausts the chain.

Can I disable fallback chains for specific models?

Yes. Set allow_fallbacks: false (the default for many models) in the model's provider configuration block. When disabled, AgentSession terminates the request immediately upon primary provider failure rather than attempting the fallback chain.

Where does the retry logic reside in the OMP source code?

The retry logic is implemented in packages/ai/src/model-thinking.ts, which coordinates with AgentSession to evaluate the fallback chain. The actual request rebuilding and provider switching occur in packages/ai/src/providers/openrouter.ts, which respects the order and allow_fallbacks fields when constructing new requests after a failure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}