How the Health Check Service Determines if a Provider Key is Healthy, Rate-Limited, or Invalid

The health check service in tashfeenahmed/freellmapi determines provider key status by decrypting stored credentials and calling provider.validateKey, assigning healthy for successful validation, invalid for explicit failures, error for transport-level exceptions, and reading rate_limited statuses that are set externally when upstream providers return HTTP 429 responses.

The health check service in the tashfeenahmed/freellmapi repository monitors API key validity through a structured validation pipeline implemented in server/src/services/health.ts. Its core routine, checkKeyHealth, orchestrates database lookups, cryptographic decryption, and provider-specific validation to classify keys into distinct health categories that drive automatic disablement logic and UI indicators.

Core Health Check Logic in server/src/services/health.ts

The validation pipeline follows a strict sequence defined in the checkKeyHealth function. Each step handles specific failure modes that determine the final status stored in the api_keys table.

Step 1: Database Retrieval and Provider Resolution

The service begins by fetching the key record from SQLite and resolving the appropriate provider for the key's platform (e.g., OpenAI, Google):

const db = getDb();
const row = db.prepare('SELECT * FROM api_keys WHERE id = ?').get(keyId);
if (!row) return 'error';

const provider = resolveProvider(row.platform, row.base_url);
if (!provider) return 'error';

Step 2: Key Decryption

The stored API key is decrypted using the row's initialization vector and authentication tag:

const apiKey = decrypt(row.encrypted_key, row.iv, row.auth_tag);

Step 3: Provider Validation

The decrypted key is passed to the provider's validateKey method with metadata including the quota pool key and endpoint:

const isValid = await provider.validateKey(apiKey, {
  platform,
  keyId,
  quotaPoolKey: inferQuotaPoolKey(row.platform, null),
  endpoint: 'models',
  origin: 'health',
});

Status Assignment Logic

The service maps validation results and exception types to four distinct statuses defined in shared/types.ts as KeyStatus = 'healthy' | 'rate_limited' | 'invalid' | 'error' | 'unknown'.

Healthy Status

When provider.validateKey returns true, the service immediately updates the database:

const status: KeyStatus = isValid ? 'healthy' : 'invalid';
db.prepare("UPDATE api_keys SET status = ?, last_checked_at = datetime('now') WHERE id = ?")
  .run(status, keyId);

Invalid Status

Explicit validation failures where validateKey returns false result in an invalid status. These are tracked in a Map<number, number> that counts consecutive failures. After three consecutive invalid results, the service automatically disables the key:

db.prepare("UPDATE api_keys SET enabled = 0 WHERE id = ?").run(keyId);

Error Status

Transport-level failures—network timeouts, DNS errors, or TLS problems—are caught by the try-catch block and recorded as error. These exceptions do not count toward the consecutive failure threshold:

catch (err) {
  db.prepare("UPDATE api_keys SET status = ?, last_checked_at = datetime('now') WHERE id = ?")
    .run('error', keyId);
  return 'error';
}

Rate-Limited Status

health.ts never sets the status to 'rate_limited'. This status is applied by external components—such as the request router or provider-specific error handlers—when a provider returns an HTTP 429 Too Many Requests response. The health service reads this status from the database and treats it as a valid but temporarily throttled key, distinct from the invalid state.

HTTP Endpoints and UI Integration

The health check logic is exposed through endpoints defined in server/src/routes/health.ts:

• GET /api/health - Returns overall system health
• POST /api/health/check/:keyId - Triggers validation for a specific key
• POST /api/health/check-all - Batch validation

The UI in client/src/pages/KeysPage.tsx renders these statuses visually: green indicators for healthy, orange for rate_limited, and red for invalid or error states.

Summary

• The checkKeyHealth function in server/src/services/health.ts drives all health determinations.
• Healthy keys result from provider.validateKey returning true.
• Invalid keys result from explicit validation failures, with automatic disablement after three consecutive failures.
• Error statuses indicate transport-level failures and are not counted toward disablement thresholds.
• Rate-limited statuses are set externally by request handlers when receiving HTTP 429 responses, not by the health check service itself.

Frequently Asked Questions

How does the health check service handle encrypted API keys?

The service retrieves the encrypted key, initialization vector, and authentication tag from the SQLite api_keys table, then calls decrypt(row.encrypted_key, row.iv, row.auth_tag) to obtain the plaintext credential before validation.

What happens when a provider key fails validation three times in a row?

The service maintains an internal Map<number, number> to track consecutive validation failures. After three consecutive invalid results, the service automatically disables the key by setting enabled = 0 in the database, preventing it from being used for new requests.

Why does the health check service mark some failures as error instead of invalid?

Transport-level failures such as network timeouts, DNS resolution errors, or TLS handshake problems are caught as exceptions and marked as error. This distinction is critical because error statuses do not increment the consecutive failure counter, whereas explicit invalid responses do.

Where does the rate_limited status originate if not from the health check?

The rate_limited status is applied by request routing logic or provider-specific error handlers elsewhere in the codebase when they receive HTTP 429 responses. The health service reads this status from the database but does not set it directly, treating rate-limited keys as temporarily restricted rather than permanently invalid.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}