How Grok2API Handles Account Failover Between Provider Accounts

Grok2API implements a pre-response failover mechanism that transparently switches to a healthy provider account when the current account returns unrecoverable HTTP 401 or 403 errors, ensuring high availability without interrupting client connections.

The Grok2API project (chenyme/grok2api) provides a gateway abstraction layer that manages multiple upstream provider accounts. When a request encounters authentication failures, rate limiting, or credential expiration, the system redirects traffic to alternative accounts before streaming begins, guaranteeing atomic response delivery.

The Account Selection Layer

Before any request reaches a provider, the Selector filters available accounts based on health, cooldown status, and quota availability.

Candidate Filtering with Acquire

The Selector.Acquire method in [backend/internal/application/gateway/selector.go](https://github.com/chenyme/grok2api/blob/main/backend/internal/application/gateway/selector.go#L33-L92) evaluates accounts by:

• Excluding accounts on active cooldown
• Validating quota availability
• Checking model capability compatibility
• Respecting sticky session keys (promptCacheKey)

The method signature accepts an excluded map parameter that prevents the selector from returning recently failed accounts during retry operations.

lease, err := selector.Acquire(
    ctx,
    account.ProviderBuild,          // Provider type identifier
    "grok-beta",                   // Target model name
    "",                            // Quota mode (empty = default)
    promptCacheKey,                // Sticky routing key
    nil,                           // Initially no exclusions
    true,                          // Allow quota probing
)
if err != nil {
    return nil, err
}
defer lease.Release()

Credential Refresh Strategy

When a provider returns HTTP 401 (Unauthorized) or 403 (Forbidden), the service first attempts to refresh the credential if the provider supports dynamic credential renewal.

The Refresh Flow

As implemented in the test suite at [backend/internal/application/gateway/service_test.go](https://github.com/chenyme/grok2api/blob/main/backend/internal/application/gateway/service_test.go#L943-L946), the service checks for Credential.Refresh support before attempting failover:

if resp.StatusCode == http.StatusUnauthorized && lease.Credential.Refresh {
    refreshed, refreshErr := providers.RefreshCredential(ctx, lease.Credential)
    if refreshErr == nil {
        lease.Credential = refreshed.Credential
        // Retry request with refreshed token on same account
    }
}

If credential refresh succeeds, the request retries on the same account. Only when refresh fails or is unsupported does the system proceed to account failover.

Pre-Response Failover Mechanics

Grok2API strictly limits failover to the pre-response phase. Once response streaming begins, the connection remains bound to the selected account to prevent splicing SSE streams across different providers.

The Failover Decision Tree

The Service.CreateResponse method (tested via the failoverAdapter in [service_test.go](https://github.com/chenyme/grok2api/blob/main/backend/internal/application/gateway/service_test.go#L69-L92)) implements the following logic:

1. Acquire initial lease via Selector.Acquire
2. Execute request through provider's ForwardResponse
3. On recoverable error: Attempt credential refresh
4. On unrecoverable error: Add account to excluded map, acquire new lease, retry

The test case TestGatewayFailsOverBeforeReturningBody validates this behavior by verifying that the adapter receives attempts on two distinct account IDs when the first returns an error:

adapter := &failoverAdapter{firstID: first.ID}
registry := provider.NewRegistry(adapter)
service := NewService(selector, registry, ...)

result, _ := service.CreateResponse(ctx, Input{Model: "grok-test"})
body, _ := io.ReadAll(result.Body)

// Verify failover occurred
if len(adapter.attempts) != 2 {
    t.Fatalf("expected failover to second account")
}
if adapter.attempts[0] == adapter.attempts[1] {
    t.Fatalf("same account used twice, failover did not occur")
}

Streaming Constraints

According to the internationalization strings in [frontend/src/shared/i18n/index.ts](https://github.com/chenyme/grok2api/blob/main/frontend/src/shared/i18n/index.ts#L955-L964), account failover only occurs before streaming begins. This architectural constraint ensures that Server-Sent Events (SSE) streams maintain consistent state and provider affinity throughout the response lifecycle.

Key Implementation Files

File	Responsibility	Source Link
selector.go	Account filtering, cooldown management, and lease acquisition	Link
service_test.go	Failover validation tests including failoverAdapter mock	Link
i18n/index.ts	User-facing documentation of pre-response failover behavior	Link

Summary

• Grok2API routes requests through a Selector that filters accounts by health, quota, and cooldown status before assignment.
• Credential refresh is attempted first on 401/403 errors if the provider supports dynamic token renewal, allowing recovery without switching accounts.
• Account failover triggers only when errors are unrecoverable, using the excluded map to prevent reselection of failed accounts.
• Pre-response guarantee ensures streaming connections (SSE) are never interrupted mid-stream by account switches, maintaining conversation state consistency.

Frequently Asked Questions

When does Grok2API trigger an account failover?

Failover triggers when the upstream provider returns HTTP 401 (Unauthorized) or 403 (Forbidden) errors that cannot be resolved by credential refresh, or when the provider indicates rate limiting that exceeds the configured retry threshold. The system marks the failed account as excluded and acquires a new lease from the Selector pool.

Does Grok2API failover during streaming responses?

No. Failover only occurs before the response body is transmitted. Once streaming begins (particularly for SSE endpoints), the connection remains bound to the originally selected account to prevent state corruption or conversation fragmentation across different provider instances.

How does Grok2API select which account to fail over to?

The Selector recalculates available candidates excluding the failed account ID from the excluded map passed to Acquire(). It prioritizes accounts with healthy credentials, available quota, and matching model capabilities, ensuring the failover target can immediately satisfy the request.

Can Grok2API refresh credentials automatically before failing over?

Yes, if the provider implementation supports the Credential.Refresh capability, Grok2API attempts to call RefreshCredential() when receiving authentication errors. Successful refresh updates the lease credential and retries the request on the same account, avoiding unnecessary failover overhead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}