How to Configure Different LLM Providers in OpenMAIC: A Complete Environment-Based Guide

OpenMAIC configures LLM providers exclusively through environment variables using the LLM_{PROVIDER}_* naming convention, dynamically loading them at startup via src/lib/server/provider-config.ts and routing all requests through the unified callLLM() client in src/lib/ai/llm.ts.

OpenMAIC abstracts large‑language‑model access behind a provider‑configuration layer that eliminates the need for source code changes when switching between OpenAI, Anthropic, Azure, or custom providers. By setting standardized environment variables, you define which providers are available, their API credentials, and supported models. This guide explains how to configure different LLM providers in OpenMAIC using the environment‑driven architecture implemented in the THU-MAIC/OpenMAIC repository.

Understanding the Provider Configuration Architecture

OpenMAIC discovers LLM providers through the provider loader located in src/lib/server/provider-config.ts. When the server initializes, the loadLLMProviders() function iterates over environment variables matching the whitelist pattern LLM_… and constructs a configuration map that is injected into the global settings store.

Validation Requirements for Usable Providers

For a provider to be marked as usable, the helper hasUsableLLMProvider enforces two conditions verified in tests/store/settings-validation.test.ts:

• An API key (or token) must be present in the corresponding environment variable.
• At least one model must be listed in the provider’s models array.

If either condition fails, the provider is excluded from the runtime map and cannot be selected for inference.

Configuring Providers via Environment Variables

Each provider requires three environment variables following the pattern LLM_{PROVIDER}_{SUFFIX}:

• LLM_{PROVIDER}_API_KEY – The authentication token.
• LLM_{PROVIDER}_BASE_URL – The API endpoint base URL.
• LLM_{PROVIDER}_MODELS – Comma‑separated list of supported model identifiers.

Example Multi‑Provider Setup

Create a .env file in the project root with the following structure:

# OpenAI Configuration

LLM_OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
LLM_OPENAI_BASE_URL=https://api.openai.com
LLM_OPENAI_MODELS=gpt-4o,gpt-4,gpt-3.5-turbo

# Anthropic Configuration

LLM_ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxx
LLM_ANTHROPIC_BASE_URL=https://api.anthropic.com
LLM_ANTHROPIC_MODELS=claude-3-5-sonnet-20240620,claude-3-opus-20240229

# Azure OpenAI (Optional)

LLM_AZURE_API_KEY=xxxxxxxxxxxxxxxxxxxx
LLM_AZURE_BASE_URL=https://my-azure-openai.openai.azure.com
LLM_AZURE_MODELS=gpt-35-turbo,gpt-4

When the server boots, loadLLMProviders() automatically discovers any subset of these configurations that satisfy the usability requirements.

Selecting Providers at Runtime

OpenMAIC supports both explicit provider selection and automatic fallback to ensure high availability.

Explicit Provider Selection

Applications can force a specific provider by passing the provider ID in the request object consumed by callLLM() in src/lib/ai/llm.ts:

import { callLLM } from '@/lib/ai/llm'

await callLLM({
  provider: 'anthropic',
  model: 'claude-3-5-sonnet-20240620',
  prompt: 'Summarize the following research paper...',
})

Automatic Fallback Logic

If the requested provider is missing or becomes unusable, the fallback logic in src/lib/server/provider-fallback.ts automatically selects the first usable provider from the loaded map. This behavior is validated in tests/server/provider-config.test.ts. The callLLM() wrapper consults this fallback chain when req.provider is omitted or invalid, ensuring the application never fails due to a single provider outage.

Extending Support to New Providers

You can add support for emerging providers like Mistral or Cohere without modifying TypeScript source files.

Define the environment variables using the standardized prefix:

# Mistral AI Configuration

LLM_MISTRAL_API_KEY=xxxxxxxxxxxxxxxx
LLM_MISTRAL_BASE_URL=https://api.mistral.ai
LLM_MISTRAL_MODELS=mistral-large,mistral-medium

The provider loader in src/lib/server/provider-config.ts automatically picks up any LLM_ prefixed variables at startup. No rebuild or code deployment is required—simply restart the server to register the new provider.

Validating Configuration Programmatically

Before executing LLM calls, verify that at least one provider is configured using the validation utilities tested in tests/store/settings-validation.test.ts:

import { hasUsableLLMProvider } from '@/store/settings'

if (!hasUsableLLMProvider(currentProviders)) {
  throw new Error('No LLM provider is configured. Please set LLM_* environment variables.')
}

This check is particularly useful for deployment scripts and health check endpoints that must validate infrastructure readiness before accepting traffic.

Summary

• Environment‑driven configuration: OpenMAIC reads LLM_{PROVIDER}_API_KEY, LLM_{PROVIDER}_BASE_URL, and LLM_{PROVIDER}_MODELS variables to build the provider map in src/lib/server/provider-config.ts.
• Zero‑code provider addition: New providers are recognized automatically when their environment variables follow the naming convention, requiring no changes to src/lib/ai/llm.ts or other core files.
• Robust fallback chain: The system automatically routes requests to the next usable provider if the primary choice is unavailable, as implemented in src/lib/server/provider-fallback.ts.
• Runtime validation: Use hasUsableLLMProvider() to programmatically confirm configuration validity before initiating expensive LLM operations.

Frequently Asked Questions

What environment variables are required to configure an LLM provider in OpenMAIC?

You must define three variables following the pattern LLM_{PROVIDER}_{SUFFIX}: LLM_{PROVIDER}_API_KEY for authentication, LLM_{PROVIDER}_BASE_URL for the API endpoint, and LLM_{PROVIDER}_MODELS as a comma‑separated list of model identifiers. The provider loader in src/lib/server/provider-config.ts skips any configuration missing either the API key or model list.

How does OpenMAIC handle missing or invalid LLM providers at runtime?

If a request specifies a provider that is missing or fails validation, the fallback logic in src/lib/server/provider-fallback.ts automatically selects the first usable provider from the loaded map. This ensures continuous operation even when primary providers experience outages, as verified by the integration tests in tests/server/provider-config.test.ts.

Can I add a new LLM provider without modifying the OpenMAIC source code?

Yes. OpenMAIC uses a dynamic discovery mechanism that recognizes any environment variables prefixed with LLM_. To add a provider like Mistral or Cohere, simply add the corresponding LLM_MISTRAL_API_KEY, LLM_MISTRAL_BASE_URL, and LLM_MISTRAL_MODELS variables to your environment and restart the server. The loadLLMProviders() function automatically registers the new entry.

Which file handles the LLM API calls after a provider is configured?

All LLM requests flow through the callLLM() function defined in src/lib/ai/llm.ts. This client resolves the concrete provider configuration, constructs the HTTP request with the appropriate authentication headers, and returns the standardized response, abstracting the differences between OpenAI, Anthropic, Azure, and other supported backends.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}