# How LMForge Securely Manages and Rotates API Keys for Multiple LLM Providers

> Discover how LMForge securely manages API keys for multiple LLM providers using environment variables and encrypted databases. Rotate keys instantly without service restarts.

- Repository: [Haohao/lmforge-end-to-end-llmops-platform-for-multi-model-agents](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents)
- Tags: security
- Published: 2026-03-03

---

**LMForge implements a two-tier credential system that stores LLM provider secrets in environment variables and application-level API keys in an encrypted PostgreSQL database, enabling instant rotation via an `is_active` flag without service restarts.**

LMForge is an open-source end-to-end LLMOps platform for multi-model agents that requires robust security for both provider credentials and client access tokens. To securely manage and rotate API keys for multiple LLM providers, the platform separates provider secrets from application authentication keys, implementing distinct storage, encryption, and lifecycle management strategies for each tier.

## The Two-Tier Credential Architecture

LMForge distinguishes between two distinct credential types to minimize blast radius and simplify rotation:

- **LLM Provider Keys**: Secrets for OpenAI, Claude, Moonshot, and Weaviate accessed via environment variables
- **Application API Keys**: Tokens issued to external clients for LMForge authentication stored in PostgreSQL with encryption

This separation ensures that rotating a client application key never affects upstream LLM provider connectivity, and vice versa.

## Securing Provider Keys via Environment Variables

LMForge loads LLM provider credentials exclusively from environment variables populated through a `.env` file at startup using `dotenv.load_dotenv()`.

According to the source code in [`api/config/config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/config.py), the configuration class reads each provider key lazily through `Config._get_env`:

```python

# api/config/config.py

self.WEAVIATE_API_KEY = _get_env("WEAVIATE_API_KEY")

```

The `_get_env` helper falls back to defaults defined in [`api/config/default_config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/default_config.py) if a variable is missing, preventing crashes while ensuring production secrets remain external. Because values are accessed only through `os.getenv()` and never logged or serialized, provider keys never appear in application logs or error traces.

To rotate a provider key, administrators update the value in the `.env` file (or secret manager) and restart the service. The new credential takes effect immediately because the configuration object reads the environment variable on each initialization.

## Encrypted Storage of Application API Keys

For client authentication, LMForge generates cryptographically secure tokens stored in the PostgreSQL `api_key` table defined in [`api/internal/model/api_key.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/model/api_key.py).

The `ApiKeyService.generate_api_key()` method in [`api/internal/service/api_key_service.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/service/api_key_service.py) (line 70) creates tokens using Python's `secrets` module:

```python

# api/internal/service/api_key_service.py

def generate_api_key(cls, api_key_prefix: str = "llmops-v1/") -> str:
    # cryptographically-secure random token

    return api_key_prefix + secrets.token_urlsafe(48)

```

The raw token is encrypted before storage (as indicated by the column comment "加密后的api秘钥" in the database schema), while only a hash or prefix returns to the client. This ensures that even database breaches do not expose usable credentials.

## Implementing Zero-Downtime API Key Rotation

LMForge enables seamless key rotation through an active/inactive status flag rather than immediate deletion. Clients rotate credentials by:

1. **Creating a new key** via `POST /openapi/api-keys` handled by [`api/internal/handler/api_key_handler.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/handler/api_key_handler.py) (lines 22-30)
2. **Deactivating the previous key** via `PATCH /openapi/api-keys/<id>/is-active` (lines 54-62)

When a client sets `is_active = false` on an existing key, the credential instantly loses access privileges without requiring service redeployment. This approach supports graceful migration scenarios where multiple keys remain valid during transition periods.

## Authentication Middleware and Validation

Every incoming request passes through `AuthenticationMiddleware` defined in [`api/internal/middleware/middleware.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/middleware/middleware.py). The middleware extracts the bearer token from the `Authorization` header and validates it against the encrypted database records:

```python

# api/internal/middleware/middleware.py (lines 33-36)

api_key = self._validate_credential(request)          # extracts header

api_key_record = self.api_key_service.get_api_by_by_credential(api_key)
if not api_key_record or not api_key_record.is_active:
    raise UnauthorizedError("Invalid or inactive API key")
return api_key_record.account   # authenticated user

```

This validation check ensures that deactivated keys cannot authenticate, providing immediate revocation capabilities. The middleware also prevents timing attacks by maintaining constant-time comparison during credential lookups.

## Summary

- LMForge uses environment variables for LLM provider secrets via [`api/config/config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/config.py), keeping credentials out of source control and logs.
- Application-level API keys are generated with `secrets.token_urlsafe(48)` and stored encrypted in PostgreSQL's `api_key` table.
- Key rotation occurs without downtime by creating new keys and setting `is_active = false` on old ones via endpoints in [`api/internal/handler/api_key_handler.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/handler/api_key_handler.py).
- The `AuthenticationMiddleware` in [`api/internal/middleware/middleware.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/middleware/middleware.py) enforces real-time validation of the `is_active` flag, enabling instant revocation.
- Provider key rotation requires only an environment variable update and service restart, while application keys rotate via API calls.

## Frequently Asked Questions

### How are LLM provider keys protected from exposure in logs?

LMForge accesses provider secrets exclusively through `os.getenv()` calls within [`api/config/config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/config.py), ensuring values are never printed, serialized, or logged. The `.env` file is ignored by Git, and production deployments load variables from secret managers rather than filesystem storage.

### Can I rotate an application API key without disrupting active requests?

Yes. Because validation occurs per-request against the database, you can create a new key via `POST /openapi/api-keys` and deactivate the old one via `PATCH /openapi/api-keys/<id>/is-active` without restarting services. Existing requests using the old key will fail only after deactivation, while new requests can immediately use the rotated credential.

### What encryption method protects stored API keys?

The `api_key` table column stores encrypted values (annotated as "加密后的api秘钥" in the model definition at [`api/internal/model/api_key.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/model/api_key.py)). While the specific algorithm isn't detailed in the handler code, the implementation ensures raw tokens are never stored in plaintext, and only cryptographic hashes or prefixes return to clients upon creation.

### How does the system handle missing provider keys without crashing?

The `Config` class in [`api/config/config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/config.py) utilizes `_get_env` with fallback defaults defined in [`api/config/default_config.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/config/default_config.py). If an environment variable is absent, the system uses the default value, preventing startup crashes while allowing production environments to override with secure secrets.