# How the Health Service in FreeLLMAPI Monitors Provider Status: Architecture and Implementation

> Discover how the FreeLLMAPI Health Service monitors LLM provider status. Learn about its architecture and implementation of scheduled health passes for automatic validation and key disabling.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: architecture
- Published: 2026-09-04

---

**The FreeLLMAPI Health Service continuously validates registered LLM provider keys through scheduled health passes that probe API credentials every 5 minutes with randomized jitter, automatically disabling keys after three consecutive invalid credential failures while distinguishing transient network errors from authentication issues.**

FreeLLMAPI is an open-source proxy server that aggregates multiple LLM provider APIs behind a unified interface. Its server-side Health Service ensures high availability by actively monitoring the validity of stored API keys against live provider endpoints. This article examines the implementation details found in the `tashfeenahmed/freellmapi` repository, tracing the flow from scheduled checks to auto-disable logic and public health endpoints.

## Scheduled Health Passes with Jitter Control

When the FreeLLMAPI server initializes, it invokes `startHealthChecker()` located in [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts) to register a recurring background job. The system uses a base probe interval of **5 minutes** defined by `CHECK_INTERVAL_MS`, but adds randomized `CHECK_INTERVAL_JITTER` to prevent thundering herd problems where multiple server instances might simultaneously bombard the same provider.

This jitter mechanism ensures that distributed deployments stagger their health checks, reducing load spikes on upstream LLM APIs. The scheduler persists until server shutdown, continuously enqueueing new health passes without blocking the main event loop.

## Concurrency and Rate Limiting

Each health pass respects strict resource constraints to avoid overwhelming providers or the local event loop. In [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts), the `checkAllKeys()` function uses `getHealthCheckConcurrency()`—defaulting to **8 parallel probes**—to limit simultaneous network requests.

Additionally, the service enforces a minimum gap between probes targeting the **same provider** using `getMinSpacingMs()`, which defaults to **1000 milliseconds**. This per-provider throttling prevents aggressive polling of a single endpoint while allowing concurrent checks across different providers.

## Key-Level Health Validation Logic

The core validation logic resides in `checkKeyHealth(keyId)` within [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts). For each key, the function executes the following steps:

1. **Database Retrieval**: Fetches the key record from the SQLite database.
2. **Provider Resolution**: Calls `resolveProvider` to instantiate the correct provider implementation.
3. **Proxy Validation**: Decrypts the stored API key and validates it through `withKeyProxy`, which mimics real traffic patterns against the provider’s actual API endpoint.

The function interprets the validation result into three distinct statuses:

- **`'healthy'`**: The provider accepted the credentials and returned a valid response.
- **`'invalid'`**: The provider rejected the credentials (HTTP 401/403). This triggers `recordInvalidFailure()`, which increments a per-key failure counter. After **three consecutive invalid failures**, the system automatically disables the key to prevent further wasted requests.
- **`'error'`**: A transport-level failure occurred (DNS resolution, timeout, or TLS error). The service records this status but **does not increment the invalid failure counter**, recognizing that network instability does not indicate bad credentials.

## Bulk Health Checks and Provider Buckets

The `checkAllKeys(opts?)` function orchestrates mass validation by gathering every enabled key and organizing them into `providerBucket` groups. This bucketing ensures that the minimum spacing rules apply per-provider rather than globally.

After executing parallel checks within the configured concurrency limits, the function returns a `HealthPassResult` object containing:

- `healthyProviders`: An array of providers where all associated keys passed validation.
- `totalProviders`: The count of distinct providers under test.
- `ratio`: The proportion of healthy providers to total providers.

This aggregation allows downstream services to quickly assess overall system health without scanning individual key records.

## Degradation Snapshots and System Integration

In [`server/src/services/degradation.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/degradation.ts), the Degradation Service consumes the `HealthPassResult` to build a `HealthSnapshot`. This snapshot stores the same three metrics—healthy providers, total providers, and ratio—in a module-level variable `lastSnapshot`.

Dashboard UIs and fallback mechanisms read this snapshot to display provider availability or trigger degraded mode when ratios fall below acceptable thresholds. The snapshot pattern decouples health monitoring from client-facing status pages, ensuring that expensive health checks do not block HTTP requests.

## Public Health Endpoint

The server exposes the health state via the `/health` route defined in [`server/src/routes/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/health.ts). This endpoint accepts optional query parameters to check a specific key ID via `checkKeyHealth()` or return the global `checkAllKeys()` results.

The route returns JSON reflecting the current `KeyStatus` enum values (`'healthy'`, `'invalid'`, `'error'`, or `'unknown'`) defined in [`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts), allowing external monitoring systems to poll for status without database access.

## Usage Examples

Trigger a manual health check for a single key:

```typescript
import { checkKeyHealth } from '@/server/src/services/health.js';

const status = await checkKeyHealth(42);
console.log(`Key 42 status: ${status}`);   // → 'healthy' | 'invalid' | 'error'

```

Run a full health pass programmatically:

```typescript
import { checkAllKeys } from '@/server/src/services/health.js';

const result = await checkAllKeys();
console.log(`Healthy providers: ${result.healthyProviders.length}/${result.totalProviders}`);

```

Query the public HTTP health endpoint:

```typescript
fetch('https://api.example.com/health')
  .then(r => r.json())
  .then(data => console.log('Health snapshot:', data));

```

## Summary

- **The Health Service initiates at server start** via `startHealthChecker()`, probing every 5 minutes with randomized jitter to prevent synchronized load spikes.
- **Concurrency is capped at 8 parallel probes** with 1000ms minimum spacing between checks targeting the same provider, protecting upstream APIs from aggressive polling.
- **Credential validation distinguishes authentication failures from network errors**, applying a three-strike auto-disable rule only to invalid credentials (401/403) while ignoring transient transport issues.
- **Bulk checks aggregate results** into `HealthPassResult` objects that the Degradation Service converts to snapshots for dashboard visualization and fallback logic.
- **The `/health` endpoint** exposes real-time status via HTTP, returning standardized `KeyStatus` values without exposing raw database connections.

## Frequently Asked Questions

### How often does FreeLLMAPI check provider health?

The Health Service runs scheduled health passes every **5 minutes** (`CHECK_INTERVAL_MS`), with added random jitter (`CHECK_INTERVAL_JITTER`) to prevent multiple instances from probing providers simultaneously. This interval balances freshness of status data against API rate limit concerns.

### What happens when a provider key fails validation?

If validation returns an authentication error (HTTP 401/403), the system calls `recordInvalidFailure()` to increment a per-key failure counter. After **three consecutive invalid failures**, the key is automatically disabled. Transport errors like timeouts or DNS failures do not count toward this limit and do not disable keys.

### How does the system handle network timeouts versus authentication errors?

The `checkKeyHealth()` function explicitly categorizes transport-level failures (DNS, timeout, TLS) as `'error'` status without incrementing the invalid failure counter. Only explicit credential rejection by the provider triggers the failure counter and potential auto-disable logic.

### Can I manually trigger a health check for a specific provider key?

Yes. Import `checkKeyHealth` from [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) and pass the key ID as an argument. You can also use the `/health` HTTP endpoint with a query parameter to check individual keys, or call `checkAllKeys()` to validate the entire keyring programmatically.