# What Is the Health Checking Service in FreeLLMAPI?

> Discover the FreeLLMAPI health checking service. It validates API keys every 5 minutes, updates their status, and disables invalid keys to ensure secure LLM requests.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-27

---

**The health checking service in FreeLLMAPI periodically validates every stored API key every 5 minutes, updates the SQLite database with `healthy`, `invalid`, or `error` statuses, and automatically disables keys after three consecutive failures to ensure only valid credentials are used for LLM requests.**

FreeLLMAPI is an open-source API gateway that aggregates multiple LLM provider keys. The **health checking service** runs as a background watchdog that maintains the integrity of the credential store by continuously verifying which keys remain active and isolating those that fail validation.

## Core Responsibilities of the Health Checking Service

The service operates as a scheduled background job that performs three critical functions: validating credentials against provider endpoints, persisting status metadata, and preventing degraded keys from entering the request pipeline.

### Periodic Validation Cycle

Every **5 minutes** (`CHECK_INTERVAL_MS`), the service queries the SQLite database for all active keys using `SELECT id, platform FROM api_keys WHERE enabled = 1`. For each record returned, it resolves the appropriate provider via `resolveProvider` and invokes the provider-specific `validateKey` routine, typically against the provider's `models` endpoint. This confirms that the API key is still authorized and functional for that platform.

### Status Tracking and Persistence

Validation results are categorized into a `KeyStatus` enum with three states: **`healthy`** (validation succeeded), **`invalid`** (explicit rejection by provider), or **`error`** (network/transport failure). The service updates the `status` column and the `last_checked_at` timestamp in [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) for every key processed, creating a real-time audit trail visible to administrators.

## Automatic Failure Handling and Key Disabling

Beyond monitoring, the service implements automated remediation by tracking failure patterns and disabling compromised keys before they impact user requests.

### The Three-Strike Failure Threshold

When a key fails validation with `isValid === false`, the service increments a per-key failure counter stored in the database. If the counter reaches **three consecutive failures** (`CONSECUTIVE_FAILURES_TO_DISABLE`), the service automatically sets `enabled = 0` for that key in the `api_keys` table, effectively removing it from the rotation pool.

### Distinguishing Transport Errors from Invalid Keys

The service differentiates between credential issues and transient infrastructure problems. Transport-level errors—including DNS failures, connection timeouts, and TLS handshake issues—are recorded as `error` status but **do not** increment the failure counter. This design prevents temporary provider outages from unnecessarily disabling valid API keys.

## Scheduler Integration and Service Lifecycle

The health checker integrates with the application's job scheduling abstraction to manage its execution lifecycle cleanly without blocking the main server thread.

### NodeScheduler Abstraction

The service relies on the `Scheduler` interface defined in [`server/src/lib/scheduler.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/scheduler.ts), with the concrete implementation `NodeScheduler` providing the execution context. This abstraction allows the health checks to run as recurring background tasks using `scheduler.every` to register the interval job.

### Starting and Stopping the Service

The `startHealthChecker(scheduler)` function accepts a `Scheduler` instance and registers the recurring validation job that invokes `checkAllKeys()`. The service can be gracefully shut down via `stopHealthChecker()`, which cleans up scheduled jobs. The entry point in [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts) bootstraps this by creating a `NodeScheduler` instance and passing it to the health checker on server startup.

## Implementation Details and Code Examples

The following patterns demonstrate how to interact with the health checking system programmatically.

### Bootstrapping the Health Checker

To initialize the service when the server starts:

```typescript
// Typical pattern from server/src/index.ts
import { NodeScheduler } from './lib/scheduler.js';
import { startHealthChecker } from './services/health.js';

const scheduler = new NodeScheduler();
startHealthChecker(scheduler);

```

### Manual Key Validation

For administrative scripts or CLI tools, validate a specific key on demand using `checkKeyHealth`:

```typescript
import { checkKeyHealth } from './services/health.js';

async function probeKey(keyId: number) {
  const status = await checkKeyHealth(keyId);
  console.log(`Key ${keyId} status → ${status}`);
}

```

### Querying Health Status from the Database

Access the latest validation states directly from the SQLite database:

```typescript
import { getDb } from '../db/index.js';

function listKeyHealth() {
  const db = getDb();
  const rows = db
    .prepare(`SELECT id, platform, status, enabled FROM api_keys`)
    .all();
  console.table(rows);
}

```

### Key Files in the Repository

- **[`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts)**: Contains `checkKeyHealth`, `checkAllKeys`, `startHealthChecker`, and `stopHealthChecker` implementations.
- **[`server/src/lib/scheduler.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/scheduler.ts)**: Defines the `Scheduler` interface and `NodeScheduler` class used for job scheduling.
- **[`server/src/services/provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/provider-quota.ts)**: Provides `inferQuotaPoolKey`, utilized during validation to attach quota metadata.
- **[`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts)**: Server entry point that instantiates the scheduler and launches the health checking service.

## Summary

- The **health checking service** runs every 5 minutes to validate all enabled API keys against their respective LLM providers.
- It updates the `status` and `last_checked_at` columns in the SQLite database, categorizing results as `healthy`, `invalid`, or `error`.
- Keys failing validation three consecutive times are automatically disabled (`enabled = 0`), while transport errors do not count toward this threshold.
- The service integrates with `NodeScheduler` in [`server/src/lib/scheduler.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/scheduler.ts) for background execution and can be controlled via `startHealthChecker` and `stopHealthChecker`.

## Frequently Asked Questions

### How often does FreeLLMAPI check the health of API keys?

The service validates all enabled keys every **5 minutes** using the `CHECK_INTERVAL_MS` constant. This interval balances the freshness of status data against API rate limit considerations and provider load.

### What happens when an API key fails validation?

A failed validation (where `isValid === false`) increments a per-key failure counter. Upon reaching **three consecutive failures** (`CONSECUTIVE_FAILURES_TO_DISABLE`), the service automatically sets the key's `enabled` flag to `0` in the database, preventing it from being selected for future requests.

### Are network timeouts counted as validation failures?

No. Transport-level errors such as **DNS failures, connection timeouts, and TLS errors** are recorded with an `error` status but explicitly do not increment the failure counter. This prevents temporary network issues from disabling otherwise valid API keys.

### Can I manually trigger a health check for a specific key?

Yes. Import `checkKeyHealth` from [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) and pass the key ID to validate a single credential on demand. This is useful for debugging or administrative verification outside the normal 5-minute cycle.