# How to Configure Different LLM Providers in OpenMAIC: A Complete Environment-Based Guide

> Easily configure various LLM providers in OpenMAIC using environment variables. This guide simplifies setup and integration for a seamless AI experience.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-03

---

**OpenMAIC configures LLM providers exclusively through environment variables using the `LLM_{PROVIDER}_*` naming convention, dynamically loading them at startup via [`src/lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-config.ts) and routing all requests through the unified `callLLM()` client in [`src/lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/ai/llm.ts).**

OpenMAIC abstracts large‑language‑model access behind a provider‑configuration layer that eliminates the need for source code changes when switching between OpenAI, Anthropic, Azure, or custom providers. By setting standardized environment variables, you define which providers are available, their API credentials, and supported models. This guide explains how to configure different LLM providers in OpenMAIC using the environment‑driven architecture implemented in the THU-MAIC/OpenMAIC repository.

## Understanding the Provider Configuration Architecture

OpenMAIC discovers LLM providers through the **provider loader** located in [`src/lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-config.ts). When the server initializes, the `loadLLMProviders()` function iterates over environment variables matching the whitelist pattern `LLM_…` and constructs a configuration map that is injected into the global settings store.

### Validation Requirements for Usable Providers

For a provider to be marked as *usable*, the helper `hasUsableLLMProvider` enforces two conditions verified in [`tests/store/settings-validation.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/store/settings-validation.test.ts):

- An **API key** (or token) must be present in the corresponding environment variable.
- At least one **model** must be listed in the provider’s `models` array.

If either condition fails, the provider is excluded from the runtime map and cannot be selected for inference.

## Configuring Providers via Environment Variables

Each provider requires three environment variables following the pattern `LLM_{PROVIDER}_{SUFFIX}`:

- `LLM_{PROVIDER}_API_KEY` – The authentication token.
- `LLM_{PROVIDER}_BASE_URL` – The API endpoint base URL.
- `LLM_{PROVIDER}_MODELS` – Comma‑separated list of supported model identifiers.

### Example Multi‑Provider Setup

Create a `.env` file in the project root with the following structure:

```dotenv

# OpenAI Configuration

LLM_OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
LLM_OPENAI_BASE_URL=https://api.openai.com
LLM_OPENAI_MODELS=gpt-4o,gpt-4,gpt-3.5-turbo

# Anthropic Configuration

LLM_ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxx
LLM_ANTHROPIC_BASE_URL=https://api.anthropic.com
LLM_ANTHROPIC_MODELS=claude-3-5-sonnet-20240620,claude-3-opus-20240229

# Azure OpenAI (Optional)

LLM_AZURE_API_KEY=xxxxxxxxxxxxxxxxxxxx
LLM_AZURE_BASE_URL=https://my-azure-openai.openai.azure.com
LLM_AZURE_MODELS=gpt-35-turbo,gpt-4

```

When the server boots, `loadLLMProviders()` automatically discovers any subset of these configurations that satisfy the usability requirements.

## Selecting Providers at Runtime

OpenMAIC supports both explicit provider selection and automatic fallback to ensure high availability.

### Explicit Provider Selection

Applications can force a specific provider by passing the `provider` ID in the request object consumed by `callLLM()` in [`src/lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/ai/llm.ts):

```typescript
import { callLLM } from '@/lib/ai/llm'

await callLLM({
  provider: 'anthropic',
  model: 'claude-3-5-sonnet-20240620',
  prompt: 'Summarize the following research paper...',
})

```

### Automatic Fallback Logic

If the requested provider is missing or becomes unusable, the fallback logic in [`src/lib/server/provider-fallback.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-fallback.ts) automatically selects the first *usable* provider from the loaded map. This behavior is validated in [`tests/server/provider-config.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/provider-config.test.ts). The `callLLM()` wrapper consults this fallback chain when `req.provider` is omitted or invalid, ensuring the application never fails due to a single provider outage.

## Extending Support to New Providers

You can add support for emerging providers like **Mistral** or **Cohere** without modifying TypeScript source files.

Define the environment variables using the standardized prefix:

```dotenv

# Mistral AI Configuration

LLM_MISTRAL_API_KEY=xxxxxxxxxxxxxxxx
LLM_MISTRAL_BASE_URL=https://api.mistral.ai
LLM_MISTRAL_MODELS=mistral-large,mistral-medium

```

The provider loader in [`src/lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-config.ts) automatically picks up any `LLM_` prefixed variables at startup. No rebuild or code deployment is required—simply restart the server to register the new provider.

## Validating Configuration Programmatically

Before executing LLM calls, verify that at least one provider is configured using the validation utilities tested in [`tests/store/settings-validation.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/store/settings-validation.test.ts):

```typescript
import { hasUsableLLMProvider } from '@/store/settings'

if (!hasUsableLLMProvider(currentProviders)) {
  throw new Error('No LLM provider is configured. Please set LLM_* environment variables.')
}

```

This check is particularly useful for deployment scripts and health check endpoints that must validate infrastructure readiness before accepting traffic.

## Summary

- **Environment‑driven configuration**: OpenMAIC reads `LLM_{PROVIDER}_API_KEY`, `LLM_{PROVIDER}_BASE_URL`, and `LLM_{PROVIDER}_MODELS` variables to build the provider map in [`src/lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-config.ts).
- **Zero‑code provider addition**: New providers are recognized automatically when their environment variables follow the naming convention, requiring no changes to [`src/lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/ai/llm.ts) or other core files.
- **Robust fallback chain**: The system automatically routes requests to the next usable provider if the primary choice is unavailable, as implemented in [`src/lib/server/provider-fallback.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-fallback.ts).
- **Runtime validation**: Use `hasUsableLLMProvider()` to programmatically confirm configuration validity before initiating expensive LLM operations.

## Frequently Asked Questions

### What environment variables are required to configure an LLM provider in OpenMAIC?

You must define three variables following the pattern `LLM_{PROVIDER}_{SUFFIX}`: `LLM_{PROVIDER}_API_KEY` for authentication, `LLM_{PROVIDER}_BASE_URL` for the API endpoint, and `LLM_{PROVIDER}_MODELS` as a comma‑separated list of model identifiers. The provider loader in [`src/lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-config.ts) skips any configuration missing either the API key or model list.

### How does OpenMAIC handle missing or invalid LLM providers at runtime?

If a request specifies a provider that is missing or fails validation, the fallback logic in [`src/lib/server/provider-fallback.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/server/provider-fallback.ts) automatically selects the first usable provider from the loaded map. This ensures continuous operation even when primary providers experience outages, as verified by the integration tests in [`tests/server/provider-config.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/provider-config.test.ts).

### Can I add a new LLM provider without modifying the OpenMAIC source code?

Yes. OpenMAIC uses a dynamic discovery mechanism that recognizes any environment variables prefixed with `LLM_`. To add a provider like Mistral or Cohere, simply add the corresponding `LLM_MISTRAL_API_KEY`, `LLM_MISTRAL_BASE_URL`, and `LLM_MISTRAL_MODELS` variables to your environment and restart the server. The `loadLLMProviders()` function automatically registers the new entry.

### Which file handles the LLM API calls after a provider is configured?

All LLM requests flow through the `callLLM()` function defined in [`src/lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/lib/ai/llm.ts). This client resolves the concrete provider configuration, constructs the HTTP request with the appropriate authentication headers, and returns the standardized response, abstracting the differences between OpenAI, Anthropic, Azure, and other supported backends.