# How to Configure Environment Variable-Based API Key Management for LLM Providers in Switchyard

> Learn to configure environment variable API key management for LLM providers in Switchyard. Securely manage keys using env_key field in launchers. Get started now.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Switchyard loads API keys from well-known environment variables such as `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, and `NVIDIA_API_KEY`, mapping them to provider configurations via the `env_key` field in launchers like [`codex_cli_launcher.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/codex_cli_launcher.py).**

Switchyard, NVIDIA's open-source LLM routing framework, uses a **convention-over-configuration** approach to API key management. Rather than storing secrets in configuration files, the system reads credentials directly from environment variables at runtime. This design keeps sensitive data out of version control while supporting multiple LLM providers through a unified interface.

## Supported Environment Variables

Switchyard recognizes provider-specific environment variables for authentication. The following table lists the well-known variables used across the codebase:

| Variable | Provider | Purpose |
|----------|----------|---------|
| `OPENAI_API_KEY` | OpenAI-compatible endpoints | API key for authentication |
| `OPENAI_BASE_URL` | OpenAI-compatible endpoints | Optional custom base URL |
| `ANTHROPIC_API_KEY` | Anthropic Claude | API key for Claude models |
| `NVIDIA_API_KEY` | NVIDIA NIM / Inference Hub | API key for NVIDIA-hosted models |
| `OPENROUTER_API_KEY` | OpenRouter | API key for OpenRouter aggregation |

These variables are documented in [`AGENTS.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/AGENTS.md) and hardcoded into launcher configurations throughout the repository.

## How the Codex Launcher Configures API Keys

The `CodexCliLauncher` class in [`switchyard/cli/launchers/codex_cli_launcher.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/codex_cli_launcher.py) demonstrates how environment variable-based key management is implemented. The launcher builds a provider configuration block that instructs the server where to find the API key.

### Key Configuration Lines

The launcher sets four critical fields (lines 73–89):

```python

# Lines 73-75: Provider identity and endpoint

f'model_providers.{_PROVIDER_ID}.name="openai"'
f'model_providers.{_PROVIDER_ID}.base_url="https://api.openai.com/v1"'

# Line 77: Force wire protocol

f'model_providers.{_PROVIDER_ID}.protocol="responses"'

# Line 81: Enable OpenAI-compatible authentication

f'model_providers.{_PROVIDER_ID}.requires_openai_auth=true'

# Line 87: Specify the environment variable containing the key

f'model_providers.{_PROVIDER_ID}.env_key="OPENAI_API_KEY"'

# Line 89: Option to disable auto-auth if needed

f'model_providers.{_PROVIDER_ID}.requires_openai_auth=false'

```

The **`env_key`** field is the critical link between the provider configuration and the runtime environment. When the server starts, it uses this field to determine which environment variable to read for the API key.

## Internal Key Retrieval Mechanism

The Rust-based LLM client (`crates/libsy-llm-client`) implements the actual key retrieval. As documented in [`crates/libsy-llm-client/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/README.md) (line 67), the client code calls:

```rust
std::env::var("OPENAI_API_KEY")

```

This direct environment lookup occurs at request time, ensuring the latest value is always used without requiring server restarts. The Python façade in [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) passes the `env_key` from the launcher configuration to the native Rust bindings.

### Authentication Flow

1. Launcher injects `env_key="OPENAI_API_KEY"` into TOML configuration
2. Server receives the configuration at startup
3. When making requests, Rust client reads `std::env::var(env_key)`
4. If `requires_openai_auth=true`, client adds `Authorization: Bearer <key>` header automatically
5. Request proceeds to the LLM provider endpoint

## Practical Configuration Examples

### Basic Shell-Based Key Management

Export the environment variable before running Switchyard:

```bash

# Set the API key for the current shell session

export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Launch with the Codex CLI launcher

switchyard launch codex --model switchyard

```

The launcher automatically detects `OPENAI_API_KEY` and configures the provider to use it.

### Custom Environment Variable Name

Override the default variable name in Python before launching:

```python
from switchyard.cli.launchers.codex_cli_launcher import CodexCliLauncher

launcher = CodexCliLauncher()

# Use a custom environment variable

launcher.provider_config["env_key"] = "MY_CUSTOM_OPENAI_KEY"

# Now the server reads from MY_CUSTOM_OPENAI_KEY instead

```

This pattern is useful when integrating with secret management systems that inject variables with non-standard names.

### Disabling Automatic Authentication

For proxy configurations that handle authentication downstream:

```python
launcher = CodexCliLauncher()
launcher.provider_config["requires_openai_auth"] = False

# The proxy must inject the Authorization header before forwarding

```

When `requires_openai_auth` is `false`, Switchyard does not add any authentication headers automatically.

### Direct Python Client Usage

```python
import os
import switchyard

# Set key programmatically (common in CI/CD environments)

os.environ["OPENAI_API_KEY"] = "sk-test-key-12345"

# Client automatically picks up the environment variable

client = switchyard.Client(provider="openai")
response = client.chat(messages=[{"role": "user", "content": "Hello, world"}])
print(response.content)

```

## Testing Environment Variable Inheritance

The test suite in [`tests/test_launchers.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tests/test_launchers.py) (lines 70–81) validates that environment variables propagate correctly through the launcher system. Tests verify:

- `OPENAI_API_KEY` is inherited from the parent process
- Default values like `"switchyard"` are injected when no environment variable is set
- The launcher configuration correctly serializes the `env_key` field

These tests ensure reliable key management across different deployment scenarios.

## Provider-Specific Configuration Patterns

### Anthropic Provider

```bash
export ANTHROPIC_API_KEY="sk-ant-api03-xxxxxxxxxxxxxxxx"
switchyard launch --provider anthropic --model claude-3-opus

```

### NVIDIA NIM Provider

```bash
export NVIDIA_API_KEY="nvapi-xxxxxxxxxxxxxxxxxxxxxxxx"
switchyard launch --provider nvidia --model meta/llama-3.1-8b-instruct

```

### OpenRouter Provider

```bash
export OPENROUTER_API_KEY="sk-or-v1-xxxxxxxxxxxxxxxx"
switchyard launch --provider openrouter --model openai/gpt-4o

```

Each provider uses the same `env_key` mechanism mapped to its respective variable name.

## Summary

- **Core mechanism**: The `env_key` field in provider configurations links to environment variable names like `OPENAI_API_KEY`
- **Primary implementation**: [`switchyard/cli/launchers/codex_cli_launcher.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/codex_cli_launcher.py) (lines 73–89) sets `env_key="OPENAI_API_KEY"` by default
- **Runtime retrieval**: Rust client in `crates/libsy-llm-client` calls `std::env::var()` using the configured key
- **Automatic headers**: Set `requires_openai_auth=true` to auto-inject `Authorization: Bearer` headers
- **Override support**: Modify `provider_config["env_key"]` to use custom variable names
- **Test coverage**: [`tests/test_launchers.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tests/test_launchers.py) verifies environment variable inheritance and default value handling

## Frequently Asked Questions

### What happens if the environment variable is not set?

Switchyard will attempt to read the variable at request time and fail with a clear error if it is missing or empty. The test suite confirms that default values like `"switchyard"` can be injected as fallbacks when configured.

### Can I use multiple API keys for different providers simultaneously?

Yes. Export all relevant variables before launching Switchyard: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `NVIDIA_API_KEY`, etc. The routing layer selects the appropriate key based on which provider handles each request.

### Is there a way to rotate API keys without restarting the server?

Since `std::env::var()` is called at request time rather than at startup, you can update the environment variable in the parent process and subsequent requests will use the new value. For containerized deployments, this typically requires restarting the container unless using orchestration-level secret rotation.

### How do I configure API keys for custom or self-hosted endpoints?

Use the `base_url` and `env_key` fields together. Set `base_url` to your endpoint and `env_key` to a custom variable name, then export that variable with your self-hosted API key.