# How FreeLLMAPI Handles Provider API Key Rotation and Health Checks

> Learn how FreeLLMAPI manages provider API key rotation and health checks. Discover their strategy for reliable LLM access using Thompson-sampled bandit scoring and automated credential validation.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-31

---

**FreeLLMAPI maintains a pool of API keys for each LLM provider and rotates them using Thompson-sampled bandit scoring for reliability, round-robin ordering for unproven keys, and automated health checks every 5 minutes that disable credentials after 3 consecutive validation failures.**

Managing multiple API keys across different LLM providers requires intelligent load distribution and fault tolerance. In the `tashfeenahmed/freellmapi` codebase, the routing and health-check subsystems work together to ensure requests always land on viable credentials while automatically removing compromised or expired keys from rotation. This article examines the specific mechanisms—from probabilistic scoring algorithms to jittered scheduling—that enable robust provider API key rotation and health checks.

## Key Selection and Rotation Strategy

The `selectKeyForModel` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines ~1430–1460) serves as the entry point for choosing which key handles an incoming request. The system implements a tiered selection strategy that balances exploitation of proven keys with exploration of newer additions.

### Thompson-Sampled Bandit Scoring

When historical reliability data exists for keys, FreeLLMAPI applies a **Thompson-sampled bandit score** via the `orderKeysByScore` helper (lines ~828–847 in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)). This algorithm assigns approximately **78% weight to reliability** metrics, ranking keys by their statistical likelihood of success. The highest-scoring key is attempted first, ensuring traffic gravitates toward the most dependable credentials while maintaining probabilistic diversity to avoid overloading single keys.

### Round-Robin Fallback and Quota Weighting

For keys without observed performance data, the system falls back to a **round-robin cursor** tracked via `roundRobinIndex`. This guarantees fair distribution across new or recently added credentials. Additionally, optional **quota weighting** through `orderKeysByRemainingQuota` can reorder the round-robin list based on remaining provider quota, preventing premature exhaustion of high-traffic keys.

## Automated Health Check System

The [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) module implements a self-healing validation layer that runs independently of the request path. This subsystem continuously monitors key validity without disrupting live traffic.

### Scheduling and Jitter

Health checks execute on a **jittered schedule** to prevent thundering herd problems against provider endpoints. The `nextHealthCheckDelayMs` function (lines ~886–890) calculates intervals using `CHECK_INTERVAL_MS` (5 minutes) plus a random variance of `CHECK_INTERVAL_JITTER` (±20%). This randomization ensures that distributed instances or restarted nodes do not synchronize their probe waves.

### Provider-Wise Interleaving and Concurrency Controls

To avoid hammering a single provider, the `runHealthPass` function (lines ~998–1060) uses `interleaveByProvider` to distribute checks evenly across different services. Concurrency is strictly limited to `DEFAULT_HEALTH_CHECK_CONCURRENCY` (8 parallel probes), with a minimum spacing of `DEFAULT_MIN_SPACING_MS` (1000 milliseconds) between requests to the same provider. These values are configurable via the `HEALTH_CHECK_CONCURRENCY` and `HEALTH_CHECK_MIN_SPACING_MS` environment variables (see `getHealthCheckConcurrency` and `getMinSpacingMs`, lines ~42–50).

### Validation Logic and Auto-Disable Thresholds

The `checkKeyHealth` function (lines ~83–130) validates each key by calling the provider’s `validateKey` method through the key’s dedicated proxy tunnel (`withKeyProxy`). The system distinguishes between credential failures and network blips:

- **Authentication errors** (401/403) increment the `consecutive‑failure counter` via `recordInvalidFailure` (lines ~64–80)
- **Transport errors** (DNS timeouts, connection resets) are logged but **do not** count toward the failure threshold

After **3 consecutive validation failures**, the key is automatically disabled (`enabled = 0`), immediately removing it from the rotation pool without manual intervention.

## Recovery and Self-Healing Mechanisms

FreeLLMAPI implements bidirectional health state management, allowing keys to recover from transient error states without administrative action.

### Automatic Re-enabling on Success

When a request successfully completes using a key previously marked with an error status, the routing layer invokes `markKeyHealthyFromRequest` (lines ~204–209 in [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts)). This function clears the error flag and resets the consecutive failure counter, effectively promoting the key back to healthy status. This design ensures that temporary network partitions or brief provider outages do not permanently blacklist otherwise valid credentials.

## Implementation Details and Code Examples

### Starting the Health Checker

Initialize the periodic validation system at server startup:

```typescript
import { startHealthChecker } from '@/services/health';
import { Scheduler } from '@/lib/scheduler';

// Scheduler wraps setTimeout/clearTimeout for jittered execution
const scheduler = new Scheduler();
startHealthChecker(scheduler);

```

### Manual Health Check Trigger

Force an immediate, non-jittered validation pass for all enabled keys:

```typescript
import { checkAllKeys } from '@/services/health';

// Useful for administrative dashboards or debugging
await checkAllKeys({ force: true });

```

### Request-Time Key Selection

The following pattern demonstrates how the router selects credentials during request handling:

```typescript
import { selectKeyForModel } from '@/services/router';
import type { RouteResult } from '@/services/router';

const route: RouteResult | null = selectKeyForModel(
  modelEntry,        // Chain/model configuration
  1024,              // Estimated token count
  new Set(),         // Keys already attempted in this request
  [],                // Diagnostic log array
);

if (!route) {
  // Return 503 or trigger fallback logic
}

```

### Marking Keys Healthy After Success

Restore a key to active rotation following a successful API call:

```typescript
import { markKeyHealthyFromRequest } from '@/services/health';

// Call after confirming the request succeeded
markKeyHealthyFromRequest(keyId);

```

## Summary

- **Statistical rotation**: FreeLLMAPI uses Thompson-sampled bandit scoring (`orderKeysByScore`) to prioritize reliable keys, falling back to round-robin for untested credentials.
- **Jittered health checks**: The system validates keys every 5 minutes with 20% random jitter, interleaving providers to distribute load.
- **Intelligent failure handling**: Only authentication errors (401/403) count toward the 3-strike auto-disable threshold; transport errors are ignored to prevent churn during network blips.
- **Self-healing recovery**: Keys automatically return to healthy status after successfully serving a live request (`markKeyHealthyFromRequest`).
- **Resource protection**: Concurrency is capped at 8 parallel checks with 1000ms minimum spacing per provider to respect rate limits.

## Frequently Asked Questions

### How does FreeLLMAPI decide which API key to use for a request?

The `selectKeyForModel` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) first checks for historical reliability data. If available, it orders keys using Thompson-sampled bandit scoring (78% reliability weight) and selects the highest scorer. For new keys without data, it uses a simple round-robin cursor, optionally weighted by remaining quota.

### What triggers an API key to be automatically disabled?

A key is disabled after **3 consecutive validation failures** recorded by `recordInvalidFailure` in [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts). Only authentication errors (401/403) increment this counter; transport-level failures like DNS timeouts or connection resets do not count toward the threshold.

### How often does FreeLLMAPI check the health of provider keys?

The health checker runs every **5 minutes** with a ±20% random jitter applied via `nextHealthCheckDelayMs` to prevent synchronized probe waves. This interval is controlled by the `CHECK_INTERVAL_MS` and `CHECK_INTERVAL_JITTER` constants.

### Can a disabled or error-marked key return to active rotation?

Yes. The `markKeyHealthyFromRequest` function automatically promotes a key back to healthy status when it successfully completes a live request. This self-healing mechanism ensures that keys disabled due to transient network issues can recover without manual administrator intervention.