# How to Configure Different LLM Providers for OpenMAIC: A Complete Guide

> Easily configure diverse LLM providers for OpenMAIC. This guide details switching between OpenAI, Anthropic, Google Gemini, and custom endpoints in the settings store.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-08-31

---

**OpenMAIC supports flexible LLM provider configuration through a three-layer architecture that lets you switch between built-in providers like OpenAI, Anthropic, and Google Gemini, or add custom OpenAI-compatible endpoints via the settings store in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts).**

OpenMAIC provides a unified interface for working with multiple large language model providers in the THU-MAIC/OpenMAIC repository. Whether you need to switch between OpenAI and Anthropic, connect to a self-hosted model gateway, or add an entirely new provider, the configuration system handles these scenarios through a persistent provider registry and dynamic configuration store.

## Understanding the Provider Architecture

OpenMAIC implements a three-layer architecture to manage LLM providers. This design separates static provider definitions from user-specific configuration and runtime execution.

### Provider Registry in [`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts)

The **Provider Registry** statically defines every built-in provider and its available models. The constant `PROVIDERS` maps provider IDs to their configurations, including default base URLs, API key requirements, and model capabilities.

```typescript
export const PROVIDERS: Record<ProviderId, ProviderConfig> = {
  openai: {
    id: 'openai',
    name: 'OpenAI',
    type: 'openai',
    defaultBaseUrl: 'https://api.openai.com/v1',
    requiresApiKey: true,
    // …model list
  },
  qwen: {
    id: 'qwen',
    name: 'Qwen',
    type: 'openai',
    defaultBaseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
    requiresApiKey: true,
    // …model list
  },
  // …other providers
};

```

Each provider entry specifies:
- `id` – Short identifier used throughout the UI and configuration (e.g., `openai`, `anthropic`, `qwen`)
- `defaultBaseUrl` – Official API endpoint for the provider
- `requiresApiKey` – Boolean indicating mandatory authentication
- `models` – Array of available models with context windows and capabilities

### Unified Configuration Store in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts)

User-specific settings persist in the **Unified Configuration Store**, a Zustand-based state manager that saves data to `localStorage` (client-side) and KV-store (account-scoped). The `providersConfig` field stores overrides for base URLs, API keys, and custom models.

The configuration shape allows you to:
- **Override the base URL** (`baseUrl`) for proxy gateways or self-hosted deployments
- **Supply API keys** (`apiKey`) per provider
- **Enable or disable providers** (`enabled`)
- **Add custom models** (`customModels`) or replace built-in definitions (`replaceBuiltInModels`)

When resolving a provider at runtime, the `getProviderConfig` function (lines 1558-1564 in [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts)) checks the static `PROVIDERS` table first, then falls back to the persisted configuration:

```typescript
function getProviderConfig(providerId: ProviderId): ProviderConfig | null {
  if (PROVIDERS[providerId]) return PROVIDERS[providerId];
  if (typeof window !== 'undefined') {
    const stored = localStorage.getItem('providersConfig');
    if (stored) {
      const cfg = JSON.parse(stored);
      const p = cfg[providerId];
      if (p) return { ...p, id: providerId };
    }
  }
  return null;
}

```

### LLM Wrapper in [`lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/llm.ts)

The **LLM Wrapper** instantiates the appropriate SDK client based on the stored configuration. When `callLLM` or `streamLLM` executes, it retrieves the provider configuration and creates the corresponding SDK instance.

According to lines 2020-2034 in [`llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/llm.ts), the wrapper switches on `cfg.type` to initialize clients:

```typescript
export async function callLLM(
  providerId: ProviderId,
  modelId: string,
  input: LLMInput,
  thinkingConfig: ThinkingConfig,
) {
  const cfg = getProviderConfig(providerId);
  if (!cfg) throw new Error(`Unknown provider ${providerId}`);

  const client = (() => {
    switch (cfg.type) {
      case 'openai':    return createOpenAI({ apiKey: cfg.apiKey, baseURL: cfg.baseUrl });
      case 'anthropic': return createAnthropic({ apiKey: cfg.apiKey, baseURL: cfg.baseUrl });
      // …other SDK creators
    }
  })();

  const body = getCompatThinkingBodyParams(providerId, modelId, thinkingConfig);
  return client.chat.completions.create({ model: modelId, messages: input.messages, ...body });
}

```

## Configuring Providers Step-by-Step

You can configure LLM providers through the Settings UI or programmatically via the store API.

### Using the Settings UI

1. Navigate to **Account → Settings** in the OpenMAIC interface
2. Select a provider from the dropdown (e.g., `openai`, `anthropic`, `qwen`, `deepseek`)
3. Enter your **API key** if the provider requires authentication
4. Toggle **Custom URL** and supply a `baseUrl` if using a proxy or self-hosted endpoint
5. Click **Add Model** to include custom models not in the built-in registry
6. Save changes to persist the configuration to `localStorage` and sync to the server KV store

The Zustand `persist` middleware automatically serializes the `providersConfig` object and syncs it across devices.

### Programmatic Configuration

For dynamic provider switching in React components, use the `useSettingsStore` hook:

```typescript
import { useSettingsStore } from '@/lib/store/settings';

// Switch to Qwen with a custom gateway
useSettingsStore.setState(state => ({
  providerId: 'qwen',
  modelId: 'qwen3.5-flash',
  providersConfig: {
    ...state.providersConfig,
    qwen: {
      ...state.providersConfig.qwen,
      baseUrl: 'https://my-qwen-proxy.example.com/v1',
      apiKey: 'sk-my-qwen-key',
    },
  },
}));

```

## Adding Custom Providers and Endpoints

OpenMAIC supports adding entirely new providers that follow the OpenAI-compatible REST contract, even if they are not defined in the static `PROVIDERS` registry.

### Injecting a Custom Provider via localStorage

You can add a custom provider by directly manipulating the `providersConfig` in `localStorage`:

```typescript
localStorage.setItem(
  'providersConfig',
  JSON.stringify({
    ...JSON.parse(localStorage.getItem('providersConfig') ?? '{}'),
    mycustom: {
      id: 'mycustom',
      name: 'My Custom LLM',
      type: 'openai',               // Uses OpenAI SDK
      defaultBaseUrl: '',
      baseUrl: 'https://my.api/v1', // Custom endpoint
      apiKey: 'sk-xxxx',
      requiresApiKey: true,
      models: [{ 
        id: 'gpt-x', 
        name: 'GPT‑X', 
        contextWindow: 1_000_000, 
        outputWindow: 128_000, 
        capabilities: { streaming: true, tools: true } 
      }],
    },
  })
);

```

After reloading the page, the new provider appears in the Settings UI. This works because `getProviderConfig` falls back to persisted configuration when a provider ID is not found in the static registry.

## Practical Code Examples

### Retrieve Current LLM Configuration

Use this React hook to access the active provider and model in your components:

```tsx
import { useSettingsStore } from '@/lib/store/settings';

export function useCurrentLLM() {
  const { providerId, modelId, providersConfig } = useSettingsStore(
    state => ({
      providerId: state.providerId,
      modelId: state.modelId,
      providersConfig: state.providersConfig,
    })
  );

  const provider = providersConfig[providerId];
  const model = provider.models.find(m => m.id === modelId);
  return { provider, model };
}

```

### Direct LLM Invocation with Custom Provider

Call a model programmatically without UI interaction:

```typescript
import { callLLM } from '@/lib/ai/llm';
import { defaultThinkingConfig } from '@/lib/ai/thinking-config';

async function generateAnswer(prompt: string) {
  const providerId = 'mycustom';
  const modelId = 'gpt-x';
  const thinkingConfig = defaultThinkingConfig;
  
  const resp = await callLLM(
    providerId, 
    modelId, 
    { messages: [{ role: 'user', content: prompt }] },
    thinkingConfig
  );
  
  return resp.choices[0].message.content;
}

```

### Adding Custom Models in Components

Extend a provider with user-defined models:

```tsx
function AddCustomModel() {
  const { providerId, providersConfig, setState } = useSettingsStore(state => ({
    providerId: state.providerId,
    providersConfig: state.providersConfig,
    setState: state.setState,
  }));

  const addModel = (id: string, name: string) => {
    const cfg = providersConfig[providerId];
    cfg.customModels = [...(cfg.customModels ?? []), { id, name }];
    setState({ providersConfig: { ...providersConfig, [providerId]: cfg } });
  };
  
  // Render form that calls addModel(...)
}

```

## Summary

- **OpenMAIC uses three layers**: Static registry ([`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts)), persistent store ([`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts)), and runtime wrapper ([`lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/llm.ts))
- **Built-in providers** include OpenAI, Anthropic, Google Gemini, Qwen, and DeepSeek, accessible via the Settings UI
- **Custom endpoints** are supported by overriding `baseUrl` in the provider configuration for proxy gateways or self-hosted models
- **New providers** can be added without code changes by injecting configuration into `localStorage.providersConfig`, provided they use the OpenAI-compatible API format
- **Configuration persists** automatically via Zustand's middleware, syncing between `localStorage` and the server-side KV store

## Frequently Asked Questions

### How do I configure a self-hosted or proxy LLM endpoint?

Override the `baseUrl` field in the provider configuration. In the Settings UI, enable the "Custom URL" toggle and enter your endpoint address (e.g., `https://my-proxy.example.com/v1`). Programmatically, update `providersConfig[providerId].baseUrl` in the store. The system uses this URL when instantiating the OpenAI SDK client in [`lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/llm.ts).

### Can I use multiple LLM providers simultaneously in one session?

Yes. While the active provider for the main chat is determined by the global `providerId` state, you can call `callLLM()` with different `providerId` arguments throughout your application. Each invocation resolves its own configuration via `getProviderConfig`, allowing you to query OpenAI for one task and Anthropic for another within the same session.

### Where are my API keys stored?

API keys are stored in the `providersConfig` object within `localStorage` (browser) and the account-scoped KV store (server). The Zustand `persist` middleware handles synchronization. Keys are never exposed to the client-side bundle hardcoded; they are injected at runtime from the configuration store when creating SDK instances in [`lib/ai/llm.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/llm.ts).

### Why isn't my custom provider appearing in the dropdown?

Ensure you have set the `type` field to `'openai'` (or another supported SDK type) and that the configuration is properly formatted JSON in `localStorage.providersConfig`. The provider must be added to the parsed configuration object before the Settings component mounts. If using the programmatic approach, verify that `getProviderConfig` returns your custom entry by checking for it in the merged configuration returned from the store.