How to Configure Multiple API Keys for Load Balancing in GPT Academic

Configure multiple API keys for load balancing in GPT Academic by providing a comma-separated list in the API_KEY variable within config.py or via environment variables; the application automatically distributes requests across valid keys using random selection.

GPT Academic (binary-husky/gpt_academic) supports simultaneous use of multiple API keys from OpenAI, Azure, API2D, and other providers. This native load-balancing capability allows you to distribute traffic across several accounts, reducing the risk of hitting rate limits on any single key. The system handles key validation, provider detection, and random selection automatically without requiring additional user code.

How Load Balancing Works in GPT Academic

The load-balancing mechanism consists of three core components that work together to manage key distribution and failure handling:

Component	Function	Source Location
API Key Definition	Reads comma-separated keys from configuration	config.py (lines 10-12)
Key Pattern Manager	Detects key types and randomly selects valid keys for each request	shared_utils/key_pattern_manager.py (lines 91-121)
Blacklist Manager	Temporarily removes failed keys from the rotation pool	request_llms/key_manager.py (lines 14-30)

When a request is initiated, the system filters the available keys to match the required provider pattern (OpenAI, Azure, etc.), then randomly selects one valid key using random.choice. This provides simple but effective round-robin style load distribution.

Step-by-Step Configuration

1. Declare Multiple Keys in config.py

Locate the API_KEY variable in config.py at the repository root. Enter your keys as a comma-separated string:

# config.py

# Multiple OpenAI / API2D / Azure keys – separate with commas

API_KEY = (
    "sk-openai-1abcdEfGhIjKlMnOpQrStUvWxYz1234567890ab,"
    "sk-openai-2abcdEfGhIjKlMnOpQrStUvWxYz0987654321cd,"
    "fk-api2d-1a2b3c-4d5e6f7g8h9i0j1k2l3m4n5o6p7q8r9s0t"
)

The comment in config.py explicitly confirms that multiple keys can be separated by commas, allowing the parser to split the string during initialization.

2. Environment Variable Alternative

You can also set the API_KEY environment variable before launching the application. Environment variables take precedence over values defined in config.py:

export API_KEY="sk-first-key...,sk-second-key...,sk-third-key..."
python main.py

This approach is particularly useful for containerized deployments or CI/CD pipelines where you want to avoid committing keys to version control.

Technical Implementation Details

Key Selection Logic

When processing a request, GPT Academic calls select_api_key() from shared_utils/key_pattern_manager.py. The function signature and logic work as follows:

# internal (simplified from shared_utils/key_pattern_manager.py)

from shared_utils.key_pattern_manager import select_api_key

api_key = select_api_key(
    llm_kwargs["api_key"],          # the whole comma-separated string

    llm_kwargs["llm_model"]           # e.g. "gpt-3.5-turbo"

)

# Returns: a randomly selected valid key for the specified model

The function splits the comma-separated string, filters keys based on the model's required provider pattern (using is_openai_api_key(), is_azure_api_key(), etc.), and returns one randomly selected valid key using random.choice. This random selection occurs on every request, providing automatic load distribution without sticky sessions.

Key Pattern Detection

The system uses regex patterns in shared_utils/key_pattern_manager.py to identify which provider each key belongs to:

• is_openai_api_key(): Detects standard OpenAI keys (sk-...)
• is_azure_api_key(): Detects Azure OpenAI keys
• is_api2d_key(): Detects API2D keys (fk-...)

This detection ensures that when you request a GPT-4 model, the system only selects from keys that can actually access GPT-4, filtering out incompatible keys automatically.

Blacklist Management

For handling failed keys, GPT Academic implements OpenAI_ApiKeyManager in request_llms/key_manager.py. This singleton class maintains a blacklist of keys that have failed repeatedly:

from request_llms.key_manager import OpenAI_ApiKeyManager

# Suppose the key "sk-openai-1abcd..." keeps failing

manager = OpenAI_ApiKeyManager()
manager.add_key_to_blacklist("sk-openai-1abcdEfGhIjKlMnOpQrStUvWxYz1234567890ab")

# Later calls that use select_avail_key() will ignore this key

The blacklist persists only for the current process session. Restarting the application clears the blacklist, allowing you to retry previously failed keys.

Advanced Usage

Custom API Key Patterns

If you are using a non-standard API key format (such as a custom proxy or third-party provider), you can define a custom regex pattern in config.py:

# config.py

CUSTOM_API_KEY_PATTERN = r"mycustom-[A-Za-z0-9]{40}"

When is_openai_api_key() runs, it will fall back to this pattern if the standard patterns don't match, allowing the key manager to recognize and balance traffic to your custom keys.

Programmatic Blacklisting

For automated failure handling, you can integrate the blacklist manager directly into custom plugins or request handlers:

from request_llms.key_manager import OpenAI_ApiKeyManager

def handle_api_error(failed_key):
    manager = OpenAI_ApiKeyManager()
    if failed_key not in manager.key_black_list:
        manager.add_key_to_blacklist(failed_key)
        print(f"Blacklisted failing key: {failed_key[:10]}...")

This approach ensures that transient failures don't repeatedly hit the same invalid key during high-volume processing sessions.

Summary

• Configuration: Provide multiple API keys as a comma-separated string in config.py via the API_KEY variable, or set the API_KEY environment variable.
• Automatic Load Balancing: The select_api_key() function in shared_utils/key_pattern_manager.py randomly selects a valid key for each request, distributing traffic across your key pool.
• Provider Detection: Built-in pattern matching (is_openai_api_key(), is_azure_api_key(), etc.) ensures only compatible keys are selected for the requested model.
• Failure Handling: The OpenAI_ApiKeyManager in request_llms/key_manager.py provides optional blacklisting to remove failing keys from the rotation until restart.
• Extensibility: Support for custom key patterns via CUSTOM_API_KEY_PATTERN allows integration with non-standard providers.

Frequently Asked Questions

Can I mix different API providers in the same configuration?

Yes. You can combine OpenAI, Azure OpenAI, API2D, and custom keys in the same comma-separated API_KEY string. The key_pattern_manager.py module automatically detects the provider type for each key using regex patterns (is_openai_api_key(), is_azure_api_key(), etc.) and filters the pool to only select keys compatible with the specific model requested.

What happens if one API key fails or gets rate limited?

If a key fails repeatedly, GPT Academic can automatically exclude it from future selections during the current session. The OpenAI_ApiKeyManager class in request_llms/key_manager.py maintains a key_black_list set. When you call add_key_to_blacklist(), the failing key is removed from the rotation pool until the application restarts, ensuring subsequent requests use only healthy keys.

How does GPT Academic choose which key to use for each request?

The selection process uses random load balancing implemented in select_api_key() within shared_utils/key_pattern_manager.py. For each request, the function splits the comma-separated key string, filters keys to match the required provider pattern for the target model, and returns one randomly selected valid key using random.choice. This ensures an even distribution of traffic across your key pool without requiring sticky sessions or manual rotation logic.

Is there a limit to how many API keys I can configure?

There is no hardcoded limit in the GPT Academic source code. You can add as many keys as needed to the comma-separated API_KEY string in config.py or the environment variable. The system parses the entire string and manages the pool dynamically. However, practical limits may apply based on your system's memory and the latency of parsing extremely long strings, though this would require an unusually large number of keys (hundreds or thousands) to become noticeable.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}