How OpenCode Handles AI Provider Fallback When One Provider Fails

OpenCode automatically retries failed LLM requests with a backup AI provider when the primary provider returns a non-200 error, excluding 404 responses, up to a maximum of three failover attempts.

OpenCode, developed by anomalyco, implements a resilient AI provider fallback system to ensure high availability for LLM requests. When you configure a fallbackProvider in your model definition, the system can automatically route traffic to an alternative provider if the primary one fails. This AI provider fallback mechanism is particularly critical for production deployments where provider outages could otherwise disrupt user sessions.

Understanding the AI Provider Fallback Architecture

The fallback system operates through a provider selector that evaluates multiple factors before routing each request. When a failure occurs, OpenCode excludes the failing provider from subsequent attempts and retries with a different option from the weighted provider pool.

Model Configuration with fallbackProvider

Each model definition in OpenCode can specify an optional fallbackProvider field that identifies which provider to use as a last resort. This configuration resides in the model schema defined in packages/console/core/src/model.ts:

// packages/console/core/src/model.ts
fallbackProvider: z.string().optional(),

The fallbackProvider must reference a provider ID that exists within the model's providers array. When all weighted primary providers fail or are excluded, the selector returns this fallback provider to ensure the request completes.

The Provider Selection Algorithm

The selectProvider() function in packages/console/app/src/routes/zen/util/handler.ts implements a deterministic selection process based on session affinity and provider weights. The function filters out disabled providers and those marked for exclusion due to recent failures, then uses a hash of the session ID to select from the remaining weighted options.

Implementation Details in handler.ts

The core fallback logic resides in the request handler utility, where OpenCode monitors HTTP responses and triggers alternative provider selection when errors occur.

The selectProvider() Function

Located in packages/console/app/src/routes/zen/util/handler.ts, this function handles the primary selection logic and final fallback resolution:

function selectProvider(...) {
  // … BYOK, trial, sticky provider checks …

  // Normal weighted selection (hash of the last 4 chars of sessionId)
  const providers = modelInfo.providers
    .filter(p => !p.disabled && !retry.excludeProviders.includes(p.id))
    .flatMap(p => Array(p.weight ?? 1).fill(p));

  // deterministic hash → index
  const index = (hash(sessionId) >>> 0) % providers.length;
  const provider = providers[index || 0];
  if (provider) return provider;

  // **fallback provider** when normal pool is exhausted
  return modelInfo.providers.find(p => p.id === modelInfo.fallbackProvider);
}

Failure Detection and Retry Logic

The retriableRequest() function executes the HTTP request and evaluates the response status. When it detects a failure that qualifies for fallback, it recursively calls itself with an updated exclusion list:

const res = await fetchWith429Retry(reqUrl, { /* … */ });

if (
  res.status !== 200 &&            // not a success
  res.status !== 404 &&            // treat 404 as a model‑not‑found error
  modelInfo.stickyProvider !== "strict" && // allow switching
  modelInfo.fallbackProvider &&   // a fallback is configured
  providerInfo.id !== modelInfo.fallbackProvider // we are not already using it
) {
  // Retry with a different provider, excluding the current one
  return retriableRequest({
    excludeProviders: [...retry.excludeProviders, providerInfo.id],
    retryCount: retry.retryCount + 1,
  });
}

MAX_FAILOVER_RETRIES Limit

OpenCode enforces a hard limit on fallback attempts to prevent infinite loops. The MAX_FAILOVER_RETRIES constant defaults to 3, meaning the system will attempt the primary provider plus up to three alternative providers before returning an error to the user.

Configuring AI Provider Fallback in OpenCode

To utilize the fallback system, you must define the provider relationships in your model configuration and understand how session policies affect provider switching.

Defining fallbackProvider in Model JSON

Model definitions reside in compiled JSON resources (Resource.ZEN_MODELS*.value). Configure the fallback by specifying the provider ID:

{
  "id": "gpt-4o-mini",
  "name": "GPT‑4o‑mini",
  "cost": { "input": 0.0005, "output": 0.0015 },
  "fallbackProvider": "openai",
  "providers": [
    { "id": "anthropic", "model": "claude-3-5-sonnet", "weight": 2 },
    { "id": "google", "model": "gemini-1.5-flash", "weight": 1 },
    { "id": "openai", "model": "gpt-4o-mini", "weight": 1 }
  ]
}

Important: The fallbackProvider must reference a provider ID that exists within the providers array.

Sticky Provider Strict Mode

When stickyProvider is set to "strict" in the model configuration, OpenCode disables the AI provider fallback mechanism for that model. This ensures all requests in a session route to the same provider, which is essential for maintaining conversation context or compliance requirements:

"stickyProvider": "strict"

According to the source code in packages/console/app/src/routes/zen/util/handler.ts, when modelInfo.stickyProvider === "strict", the fallback condition evaluates to false, preventing provider switching even during outages.

Code Example: Complete Fallback Flow

Consider a scenario where a model is configured with Anthropic as the primary weighted provider, Google as secondary, and OpenAI as the fallback:

// Assume a model config like:
// {
//   "id": "gpt-4o-mini",
//   "fallbackProvider": "openai",
//   "providers": [{ "id": "anthropic", ... }, { "id": "google", ... }, { "id": "openai", ... }]
// }

// 1️⃣ Primary request goes to Anthropic (highest weight).
// 2️⃣ Anthropic returns 500 (service down).
// 3️⃣ OpenCode detects the failure, excludes "anthropic", and retries.
// 4️⃣ The weighted selection now picks Google (if still available) …
// 5️⃣ If all weighted providers fail, the selector finally returns the
//    "openai" fallback provider and the request is sent there.

This demonstrates how OpenCode's AI provider fallback ensures request completion even during multi-provider outages, while respecting the configured MAX_FAILOVER_RETRIES limit of three attempts.

Summary

• OpenCode implements AI provider fallback through a declarative fallbackProvider field in the model configuration, located in packages/console/core/src/model.ts.
• The selectProvider() function in packages/console/app/src/routes/zen/util/handler.ts handles weighted provider selection and final fallback resolution when the primary pool is exhausted.
• Failed requests trigger the retriableRequest() logic, which excludes the failing provider and retries up to MAX_FAILOVER_RETRIES (3) times.
• The system treats 404 errors as model-not-found conditions that do not trigger fallback, while other non-200 status codes initiate the retry process.
• Setting stickyProvider to "strict" disables AI provider fallback for that model, ensuring session consistency at the cost of availability during outages.

Frequently Asked Questions

What triggers an AI provider fallback in OpenCode?

An AI provider fallback triggers when a request returns a non-200 HTTP status code that is not 404, the model has a fallbackProvider configured, and the current provider is not already the fallback. The system also checks that stickyProvider is not set to "strict", which would prevent provider switching regardless of errors.

How many retry attempts does OpenCode allow for provider fallback?

OpenCode allows up to three retry attempts for provider fallback, controlled by the MAX_FAILOVER_RETRIES constant defined in packages/console/app/src/routes/zen/util/handler.ts. This means the system can attempt the primary provider plus up to three alternative providers before returning an error to the user.

Can I disable AI provider fallback for specific models?

Yes, you can disable AI provider fallback for specific models by setting the stickyProvider field to "strict" in the model configuration JSON. When this mode is enabled, OpenCode will not switch providers even if the current provider returns errors, ensuring session consistency but sacrificing automatic failover capabilities.

Where is the fallback provider configured in OpenCode?

The fallback provider is configured in the model definition JSON files (referenced as Resource.ZEN_MODELS*.value in the codebase) and defined in the schema at packages/console/core/src/model.ts. Each model can specify an optional fallbackProvider string that must match a provider ID listed in the model's providers array, as implemented in the selectProvider() function within packages/console/app/src/routes/zen/util/handler.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}