How to Configure Environment Variable-Based API Key Management for LLM Providers in Switchyard

Switchyard loads API keys from well-known environment variables such as OPENAI_API_KEY, ANTHROPIC_API_KEY, and NVIDIA_API_KEY, mapping them to provider configurations via the env_key field in launchers like codex_cli_launcher.py.

Switchyard, NVIDIA's open-source LLM routing framework, uses a convention-over-configuration approach to API key management. Rather than storing secrets in configuration files, the system reads credentials directly from environment variables at runtime. This design keeps sensitive data out of version control while supporting multiple LLM providers through a unified interface.

Supported Environment Variables

Switchyard recognizes provider-specific environment variables for authentication. The following table lists the well-known variables used across the codebase:

Variable	Provider	Purpose
OPENAI_API_KEY	OpenAI-compatible endpoints	API key for authentication
OPENAI_BASE_URL	OpenAI-compatible endpoints	Optional custom base URL
ANTHROPIC_API_KEY	Anthropic Claude	API key for Claude models
NVIDIA_API_KEY	NVIDIA NIM / Inference Hub	API key for NVIDIA-hosted models
OPENROUTER_API_KEY	OpenRouter	API key for OpenRouter aggregation

These variables are documented in AGENTS.md and hardcoded into launcher configurations throughout the repository.

How the Codex Launcher Configures API Keys

The CodexCliLauncher class in switchyard/cli/launchers/codex_cli_launcher.py demonstrates how environment variable-based key management is implemented. The launcher builds a provider configuration block that instructs the server where to find the API key.

Key Configuration Lines

The launcher sets four critical fields (lines 73–89):

# Lines 73-75: Provider identity and endpoint

f'model_providers.{_PROVIDER_ID}.name="openai"'
f'model_providers.{_PROVIDER_ID}.base_url="https://api.openai.com/v1"'

# Line 77: Force wire protocol

f'model_providers.{_PROVIDER_ID}.protocol="responses"'

# Line 81: Enable OpenAI-compatible authentication

f'model_providers.{_PROVIDER_ID}.requires_openai_auth=true'

# Line 87: Specify the environment variable containing the key

f'model_providers.{_PROVIDER_ID}.env_key="OPENAI_API_KEY"'

# Line 89: Option to disable auto-auth if needed

f'model_providers.{_PROVIDER_ID}.requires_openai_auth=false'

The env_key field is the critical link between the provider configuration and the runtime environment. When the server starts, it uses this field to determine which environment variable to read for the API key.

Internal Key Retrieval Mechanism

The Rust-based LLM client (crates/libsy-llm-client) implements the actual key retrieval. As documented in crates/libsy-llm-client/README.md (line 67), the client code calls:

std::env::var("OPENAI_API_KEY")

This direct environment lookup occurs at request time, ensuring the latest value is always used without requiring server restarts. The Python façade in switchyard_rust/libsy.py passes the env_key from the launcher configuration to the native Rust bindings.

Authentication Flow

1. Launcher injects env_key="OPENAI_API_KEY" into TOML configuration
2. Server receives the configuration at startup
3. When making requests, Rust client reads std::env::var(env_key)
4. If requires_openai_auth=true, client adds Authorization: Bearer <key> header automatically
5. Request proceeds to the LLM provider endpoint

Practical Configuration Examples

Basic Shell-Based Key Management

Export the environment variable before running Switchyard:

# Set the API key for the current shell session

export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Launch with the Codex CLI launcher

switchyard launch codex --model switchyard

The launcher automatically detects OPENAI_API_KEY and configures the provider to use it.

Custom Environment Variable Name

Override the default variable name in Python before launching:

from switchyard.cli.launchers.codex_cli_launcher import CodexCliLauncher

launcher = CodexCliLauncher()

# Use a custom environment variable

launcher.provider_config["env_key"] = "MY_CUSTOM_OPENAI_KEY"

# Now the server reads from MY_CUSTOM_OPENAI_KEY instead

This pattern is useful when integrating with secret management systems that inject variables with non-standard names.

Disabling Automatic Authentication

For proxy configurations that handle authentication downstream:

launcher = CodexCliLauncher()
launcher.provider_config["requires_openai_auth"] = False

# The proxy must inject the Authorization header before forwarding

When requires_openai_auth is false, Switchyard does not add any authentication headers automatically.

Direct Python Client Usage

import os
import switchyard

# Set key programmatically (common in CI/CD environments)

os.environ["OPENAI_API_KEY"] = "sk-test-key-12345"

# Client automatically picks up the environment variable

client = switchyard.Client(provider="openai")
response = client.chat(messages=[{"role": "user", "content": "Hello, world"}])
print(response.content)

Testing Environment Variable Inheritance

The test suite in tests/test_launchers.py (lines 70–81) validates that environment variables propagate correctly through the launcher system. Tests verify:

• OPENAI_API_KEY is inherited from the parent process
• Default values like "switchyard" are injected when no environment variable is set
• The launcher configuration correctly serializes the env_key field

These tests ensure reliable key management across different deployment scenarios.

Provider-Specific Configuration Patterns

Anthropic Provider

export ANTHROPIC_API_KEY="sk-ant-api03-xxxxxxxxxxxxxxxx"
switchyard launch --provider anthropic --model claude-3-opus

NVIDIA NIM Provider

export NVIDIA_API_KEY="nvapi-xxxxxxxxxxxxxxxxxxxxxxxx"
switchyard launch --provider nvidia --model meta/llama-3.1-8b-instruct

OpenRouter Provider

export OPENROUTER_API_KEY="sk-or-v1-xxxxxxxxxxxxxxxx"
switchyard launch --provider openrouter --model openai/gpt-4o

Each provider uses the same env_key mechanism mapped to its respective variable name.

Summary

• Core mechanism: The env_key field in provider configurations links to environment variable names like OPENAI_API_KEY
• Primary implementation: switchyard/cli/launchers/codex_cli_launcher.py (lines 73–89) sets env_key="OPENAI_API_KEY" by default
• Runtime retrieval: Rust client in crates/libsy-llm-client calls std::env::var() using the configured key
• Automatic headers: Set requires_openai_auth=true to auto-inject Authorization: Bearer headers
• Override support: Modify provider_config["env_key"] to use custom variable names
• Test coverage: tests/test_launchers.py verifies environment variable inheritance and default value handling

Frequently Asked Questions

What happens if the environment variable is not set?

Switchyard will attempt to read the variable at request time and fail with a clear error if it is missing or empty. The test suite confirms that default values like "switchyard" can be injected as fallbacks when configured.

Can I use multiple API keys for different providers simultaneously?

Yes. Export all relevant variables before launching Switchyard: OPENAI_API_KEY, ANTHROPIC_API_KEY, NVIDIA_API_KEY, etc. The routing layer selects the appropriate key based on which provider handles each request.

Is there a way to rotate API keys without restarting the server?

Since std::env::var() is called at request time rather than at startup, you can update the environment variable in the parent process and subsequent requests will use the new value. For containerized deployments, this typically requires restarting the container unless using orchestration-level secret rotation.

How do I configure API keys for custom or self-hosted endpoints?

Use the base_url and env_key fields together. Set base_url to your endpoint and env_key to a custom variable name, then export that variable with your self-hosted API key.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}