How LMForge Securely Manages and Rotates API Keys for Multiple LLM Providers

LMForge implements a two-tier credential system that stores LLM provider secrets in environment variables and application-level API keys in an encrypted PostgreSQL database, enabling instant rotation via an is_active flag without service restarts.

LMForge is an open-source end-to-end LLMOps platform for multi-model agents that requires robust security for both provider credentials and client access tokens. To securely manage and rotate API keys for multiple LLM providers, the platform separates provider secrets from application authentication keys, implementing distinct storage, encryption, and lifecycle management strategies for each tier.

The Two-Tier Credential Architecture

LMForge distinguishes between two distinct credential types to minimize blast radius and simplify rotation:

• LLM Provider Keys: Secrets for OpenAI, Claude, Moonshot, and Weaviate accessed via environment variables
• Application API Keys: Tokens issued to external clients for LMForge authentication stored in PostgreSQL with encryption

This separation ensures that rotating a client application key never affects upstream LLM provider connectivity, and vice versa.

Securing Provider Keys via Environment Variables

LMForge loads LLM provider credentials exclusively from environment variables populated through a .env file at startup using dotenv.load_dotenv().

According to the source code in api/config/config.py, the configuration class reads each provider key lazily through Config._get_env:

# api/config/config.py

self.WEAVIATE_API_KEY = _get_env("WEAVIATE_API_KEY")

The _get_env helper falls back to defaults defined in api/config/default_config.py if a variable is missing, preventing crashes while ensuring production secrets remain external. Because values are accessed only through os.getenv() and never logged or serialized, provider keys never appear in application logs or error traces.

To rotate a provider key, administrators update the value in the .env file (or secret manager) and restart the service. The new credential takes effect immediately because the configuration object reads the environment variable on each initialization.

Encrypted Storage of Application API Keys

For client authentication, LMForge generates cryptographically secure tokens stored in the PostgreSQL api_key table defined in api/internal/model/api_key.py.

The ApiKeyService.generate_api_key() method in api/internal/service/api_key_service.py (line 70) creates tokens using Python's secrets module:

# api/internal/service/api_key_service.py

def generate_api_key(cls, api_key_prefix: str = "llmops-v1/") -> str:
    # cryptographically-secure random token

    return api_key_prefix + secrets.token_urlsafe(48)

The raw token is encrypted before storage (as indicated by the column comment "加密后的api秘钥" in the database schema), while only a hash or prefix returns to the client. This ensures that even database breaches do not expose usable credentials.

Implementing Zero-Downtime API Key Rotation

LMForge enables seamless key rotation through an active/inactive status flag rather than immediate deletion. Clients rotate credentials by:

1. Creating a new key via POST /openapi/api-keys handled by api/internal/handler/api_key_handler.py (lines 22-30)
2. Deactivating the previous key via PATCH /openapi/api-keys/<id>/is-active (lines 54-62)

When a client sets is_active = false on an existing key, the credential instantly loses access privileges without requiring service redeployment. This approach supports graceful migration scenarios where multiple keys remain valid during transition periods.

Authentication Middleware and Validation

Every incoming request passes through AuthenticationMiddleware defined in api/internal/middleware/middleware.py. The middleware extracts the bearer token from the Authorization header and validates it against the encrypted database records:

# api/internal/middleware/middleware.py (lines 33-36)

api_key = self._validate_credential(request)          # extracts header

api_key_record = self.api_key_service.get_api_by_by_credential(api_key)
if not api_key_record or not api_key_record.is_active:
    raise UnauthorizedError("Invalid or inactive API key")
return api_key_record.account   # authenticated user

This validation check ensures that deactivated keys cannot authenticate, providing immediate revocation capabilities. The middleware also prevents timing attacks by maintaining constant-time comparison during credential lookups.

Summary

• LMForge uses environment variables for LLM provider secrets via api/config/config.py, keeping credentials out of source control and logs.
• Application-level API keys are generated with secrets.token_urlsafe(48) and stored encrypted in PostgreSQL's api_key table.
• Key rotation occurs without downtime by creating new keys and setting is_active = false on old ones via endpoints in api/internal/handler/api_key_handler.py.
• The AuthenticationMiddleware in api/internal/middleware/middleware.py enforces real-time validation of the is_active flag, enabling instant revocation.
• Provider key rotation requires only an environment variable update and service restart, while application keys rotate via API calls.

Frequently Asked Questions

How are LLM provider keys protected from exposure in logs?

LMForge accesses provider secrets exclusively through os.getenv() calls within api/config/config.py, ensuring values are never printed, serialized, or logged. The .env file is ignored by Git, and production deployments load variables from secret managers rather than filesystem storage.

Can I rotate an application API key without disrupting active requests?

Yes. Because validation occurs per-request against the database, you can create a new key via POST /openapi/api-keys and deactivate the old one via PATCH /openapi/api-keys/<id>/is-active without restarting services. Existing requests using the old key will fail only after deactivation, while new requests can immediately use the rotated credential.

What encryption method protects stored API keys?

The api_key table column stores encrypted values (annotated as "加密后的api秘钥" in the model definition at api/internal/model/api_key.py). While the specific algorithm isn't detailed in the handler code, the implementation ensures raw tokens are never stored in plaintext, and only cryptographic hashes or prefixes return to clients upon creation.

How does the system handle missing provider keys without crashing?

The Config class in api/config/config.py utilizes _get_env with fallback defaults defined in api/config/default_config.py. If an environment variable is absent, the system uses the default value, preventing startup crashes while allowing production environments to override with secure secrets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}