How the ClosedClaw Gateway Manages Model Fallback When the Primary Ollama Model Becomes Unavailable

The ClosedClaw gateway automatically routes requests to alternative models when the primary Ollama instance fails, attempting each candidate in priority order until one succeeds or all options are exhausted.

The ClosedClaw gateway, an open-source AI orchestration layer, implements resilient model fallback to ensure high availability when integrating with local Ollama instances. When the primary model becomes unreachable due to daemon failures, network timeouts, or unknown model names, the gateway manages model fallback by cycling through a configured priority list of alternative providers. This architecture prevents single points of failure in agent-driven workflows.

Understanding the Model Fallback Architecture

The fallback system is centralized in src/agents/model-fallback.ts and exposes the runWithModelFallback function. This helper orchestrates the entire lifecycle of candidate resolution, authentication, and error aggregation.

Candidate Resolution Strategy

Before executing any model, the gateway builds a prioritized candidate list via resolveFallbackCandidates. This function:

• Loads the global default model from agents.defaults.model in the configuration schema (src/config/schema.ts)
• Parses the fallbacks array defined in agents.defaults.model.fallbacks, converting each string into a structured {provider, model} pair using resolveModelRefFromString
• Prepends the primary model as the first candidate unless fallbacksOverride is provided, in which case the override list replaces the global configuration entirely

Authentication and Rate Limit Management

For each candidate, the gateway must resolve authentication credentials and respect rate-limit cooldowns. The runWithModelFallback function creates an authStore instance and calls resolveAuthProfileOrder to retrieve valid profiles for the candidate's provider.

If all authentication profiles for a specific provider are currently in cooldown due to previous rate-limit errors, the candidate is skipped entirely and recorded as a rate-limit failure. This prevents the gateway from wasting time on providers that are known to be throttled.

How the Gateway Executes Model Fallback

Once candidates are resolved and authenticated, the gateway enters the execution loop. This process handles the actual model invocation, error classification, and audit trail generation.

The runWithModelFallback Implementation

The core execution flow is implemented in src/agents/model-fallback.ts and invoked by entry points such as src/commands/agent.ts and the auto-reply runners in src/auto-reply/reply/. The function accepts a run callback that performs the actual provider-specific request:

// src/commands/agent.ts (excerpt)
const result = await runWithModelFallback({
  cfg,
  provider: "Ollama",
  model: "qwen3:8b",               // primary model from config
  run: async (provider, model) => {
    // Real call to Ollama HTTP API or other provider
    return runEmbeddedPiAgent({ provider, model, ... });
  },
  onError: ({ provider, model, error, attempt, total }) => {
    context.log.warn(
      `Attempt ${attempt}/${total} failed for ${provider}/${model}: ${error}`,
    );
  },
});

The runWithModelFallback function iterates through the candidate list, invoking the run callback for each. It awaits the result and, upon success, returns an object containing:

• result: The successful model output
• provider and model: The winning candidate identifiers
• attempts: A full audit trail of all failed tries with error details

Error Classification and Audit Trails

When a candidate fails, the gateway applies sophisticated error handling logic:

1. AbortError Handling: If the error is an AbortError and not a timeout, it is immediately re-thrown. This ensures that user-initiated cancellations propagate correctly without triggering fallback to alternative models.

2. FailoverError Coercion: All other errors are coerced to a FailoverError type using coerceToFailoverError. If the error cannot be classified as a failover type, the original error bubbles up immediately.

3. Metadata Collection: For valid failover errors, the gateway records:

   • provider and model identifiers
   • Error message
   • Failure reason (rate_limit, timeout, auth_error, etc.)
   • HTTP status code
   • Provider-specific error code

This metadata populates the attempts array returned upon success, enabling complete observability into which models failed and why.

Exhaustion and Final Error Reporting

When no candidate succeeds, the gateway throws a comprehensive aggregation error:

All models failed (3): ollama/qwen-8b: timeout (timeout) | openai/gpt-4: rate limit (rate_limit) | anthropic/claude-3: auth error (auth_error)

This error preserves the original failure as the cause property, allowing upstream handlers to inspect the root failure while presenting a clear summary to operators.

Configuration and Integration Points

The fallback behavior is controlled through the configuration schema defined in src/config/schema.ts and default values in src/config/defaults.ts.

Defining Fallback Models

Operators configure fallback chains via the agents.defaults.model.fallbacks array:

// Example configuration structure
{
  "agents": {
    "defaults": {
      "model": {
        "provider": "Ollama",
        "model": "qwen3:8b",
        "fallbacks": [
          "openai/gpt-4.1-mini",
          "anthropic/claude-3-haiku",
          "ollama/llama3:70b"
        ]
      }
    }
  }
}

The resolveModelRefFromString function parses these strings, expanding aliases and applying the default provider when not explicitly specified.

CLI and Auto-Reply Integration

The src/commands/agent.ts file demonstrates how the CLI command wraps its execution:

const result = await runWithModelFallback({
  cfg,
  provider: "Ollama",
  model: "qwen3:8b",
  run: async (provider, model) => {
    return runEmbeddedPiAgent({ provider, model, /* ... */ });
  },
});

Similarly, auto-reply runners in src/auto-reply/reply/agent-runner.response-usage-footer.ts and related files use the same wrapper to ensure that background agent processes remain resilient to Ollama outages.

Summary

• The ClosedClaw gateway manages model fallback through the runWithModelFallback function in src/agents/model-fallback.ts, which orchestrates candidate resolution, authentication, and error aggregation.
• When the primary Ollama model fails, the gateway automatically cycles through the agents.defaults.model.fallbacks list, attempting each provider in priority order.
• The system distinguishes between fatal errors (like user aborts) and transient failover errors (like timeouts or rate limits), collecting detailed metadata for each failed attempt.
• If all candidates fail, the gateway throws a comprehensive error summarizing every attempt; otherwise, it returns the successful result along with a complete audit trail of previous failures.

Frequently Asked Questions

How does the gateway determine which fallback model to try first?

The gateway builds a prioritized candidate list via resolveFallbackCandidates in src/agents/model-fallback.ts. It always places the primary model first, followed by the models listed in the agents.defaults.model.fallbacks configuration array. If a request provides a fallbacksOverride parameter, that list replaces the global configuration entirely, with the primary model appended automatically.

What happens if all fallback models are rate limited?

If all authentication profiles for a specific provider are in cooldown due to previous rate-limit errors, the gateway skips that candidate entirely and records it as a rate-limit failure. If every candidate across all providers is either rate-limited or otherwise failing, the gateway exhausts the list and throws an aggregated error such as All models failed (3): ollama/qwen-7b: rate limit (rate_limit) | openai/gpt-4: rate limit (rate_limit).

Does the gateway support overriding fallbacks for specific requests?

Yes. The runWithModelFallback function accepts an optional fallbacksOverride parameter. When provided, this array replaces the global agents.defaults.model.fallbacks configuration for that specific execution. The primary model is automatically appended to the override list, ensuring it is attempted first unless explicitly excluded.

How are authentication errors handled during fallback attempts?

For each candidate, the gateway calls resolveAuthProfileOrder to retrieve valid authentication profiles for that provider. If authentication fails, the error is coerced to a FailoverError with the reason set to auth_error. This failure is logged and added to the attempts array, allowing the gateway to proceed to the next candidate. Only if all candidates fail due to authentication or other errors does the gateway raise the final aggregated failure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}