How LLM Configuration Is Managed for Different Providers Like OpenAI in Mirofish

The mirofish repository centralizes LLM settings in environment variables through a Config class, enabling seamless provider switching by updating LLM_BASE_URL and LLM_MODEL_NAME without modifying application code.

The mirofish project implements a provider-agnostic architecture for LLM configuration management, allowing developers to integrate with OpenAI, Azure OpenAI, or any OpenAI-compatible endpoint through a unified interface. This approach decouples provider-specific credentials and endpoints from business logic, ensuring that swapping between LLM services requires only environment variable updates rather than code changes.

Environment-Based Configuration Architecture

All LLM configuration settings are centralized in the Config class located in backend/app/config.py. At application startup, this class reads values from the project-root .env file or falls back to process environment variables when the file is absent.

The configuration exposes three critical environment variables:

# backend/app/config.py

LLM_API_KEY = os.environ.get('LLM_API_KEY')
LLM_BASE_URL = os.environ.get('LLM_BASE_URL', 'https://api.openai.com/v1')
LLM_MODEL_NAME = os.environ.get('LLM_MODEL_NAME', 'gpt-4o-mini')

The default values point to the public OpenAI API, but any OpenAI-compatible endpoint can be substituted by overriding LLM_BASE_URL. This includes Azure OpenAI Service, self-hosted models, or third-party providers implementing the OpenAI HTTP specification.

The LLMClient Wrapper

The LLMClient class in backend/app/utils/llm_client.py creates a single OpenAI client instance using values from Config or explicitly passed parameters:

# backend/app/utils/llm_client.py

self.client = OpenAI(api_key=self.api_key, base_url=self.base_url)

Because the underlying SDK is the official openai package, any provider implementing the OpenAI API specification works without code modifications. The wrapper exposes two primary methods:

• LLMClient.chat() – Standard text completion
• LLMClient.chat_json() – Structured JSON output parsing

Application code interacts with the LLM solely through these methods, never touching API keys or URLs directly. This guarantees that swapping providers is purely a configuration concern.

Configuration Validation

The Config class implements a validate() method that checks for required variables at startup:

# backend/app/config.py

errors = Config.validate()

This validation ensures that LLM_API_KEY and ZEP_API_KEY are present, helping catch misconfiguration early in the deployment pipeline before the application attempts to initialize LLM connections.

Practical Implementation Examples

Setting Up Provider Credentials

Define provider settings in the .env file or CI/CD secret store:

# .env

LLM_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
LLM_BASE_URL=https://<your-provider>.openai.azure.com/v1
LLM_MODEL_NAME=gpt-4o-mini

The repository includes an .env.example file at the project root listing these variables for reference.

Standard Client Instantiation

Import and instantiate LLMClient to use default configuration values:

from backend.app.utils.llm_client import LLMClient

client = LLMClient()

response = client.chat(
    messages=[{"role": "user", "content": "Explain quantum entanglement in plain language."}]
)
print(response)

Runtime Provider Override

Override settings for individual requests without modifying environment files:

client = LLMClient(
    api_key="sk-xxxx",
    base_url="https://custom.api/v1",
    model="gpt-4o-mini"
)

json_result = client.chat_json(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Provide a JSON summary of the following text."}
    ]
)

Startup Validation

Validate configuration before application initialization:

from backend.app.config import Config

errors = Config.validate()
if errors:
    raise RuntimeError(f"Configuration errors: {errors}")

Summary

• Centralized configuration lives in backend/app/config.py, reading from environment variables with sensible defaults for OpenAI.
• Provider-agnostic design allows switching to Azure OpenAI, local models, or compatible services by updating LLM_BASE_URL and LLM_MODEL_NAME.
• Typed wrapper in backend/app/utils/llm_client.py encapsulates the openai SDK, exposing chat() and chat_json() methods.
• Environment-driven approach ensures no code changes are required when swapping between LLM providers.
• Early validation via Config.validate() prevents runtime errors from missing credentials.

Frequently Asked Questions

How do I switch from OpenAI to Azure OpenAI Service?

Update the LLM_BASE_URL environment variable to your Azure OpenAI endpoint (e.g., https://<your-resource>.openai.azure.com/v1) and set LLM_API_KEY to your Azure API key. The LLMClient requires no code changes because Azure OpenAI implements the OpenAI HTTP specification.

What environment variables are required to configure the LLM?

The three primary variables are LLM_API_KEY (authentication), LLM_BASE_URL (endpoint URL, defaults to OpenAI), and LLM_MODEL_NAME (model identifier, defaults to gpt-4o-mini). Additionally, ZEP_API_KEY is required for full application functionality according to the validation logic in backend/app/config.py.

Can I use multiple LLM providers simultaneously in the same application?

Yes. While the default LLMClient uses global configuration, you can instantiate multiple clients with different api_key, base_url, and model parameters passed directly to the constructor. Each instance maintains its own connection to a specific provider.

How does the application validate LLM configuration at startup?

The Config.validate() method checks for the presence of required environment variables (LLM_API_KEY and ZEP_API_KEY) and returns a list of error messages if any are missing. This validation should be called during application initialization to fail fast on misconfiguration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}