How to Configure Different LLM Providers for OpenMAIC: A Complete Guide

OpenMAIC supports flexible LLM provider configuration through a three-layer architecture that lets you switch between built-in providers like OpenAI, Anthropic, and Google Gemini, or add custom OpenAI-compatible endpoints via the settings store in lib/store/settings.ts.

OpenMAIC provides a unified interface for working with multiple large language model providers in the THU-MAIC/OpenMAIC repository. Whether you need to switch between OpenAI and Anthropic, connect to a self-hosted model gateway, or add an entirely new provider, the configuration system handles these scenarios through a persistent provider registry and dynamic configuration store.

Understanding the Provider Architecture

OpenMAIC implements a three-layer architecture to manage LLM providers. This design separates static provider definitions from user-specific configuration and runtime execution.

Provider Registry in lib/ai/providers.ts

The Provider Registry statically defines every built-in provider and its available models. The constant PROVIDERS maps provider IDs to their configurations, including default base URLs, API key requirements, and model capabilities.

export const PROVIDERS: Record<ProviderId, ProviderConfig> = {
  openai: {
    id: 'openai',
    name: 'OpenAI',
    type: 'openai',
    defaultBaseUrl: 'https://api.openai.com/v1',
    requiresApiKey: true,
    // …model list
  },
  qwen: {
    id: 'qwen',
    name: 'Qwen',
    type: 'openai',
    defaultBaseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
    requiresApiKey: true,
    // …model list
  },
  // …other providers
};

Each provider entry specifies:

• id – Short identifier used throughout the UI and configuration (e.g., openai, anthropic, qwen)
• defaultBaseUrl – Official API endpoint for the provider
• requiresApiKey – Boolean indicating mandatory authentication
• models – Array of available models with context windows and capabilities

Unified Configuration Store in lib/store/settings.ts

User-specific settings persist in the Unified Configuration Store, a Zustand-based state manager that saves data to localStorage (client-side) and KV-store (account-scoped). The providersConfig field stores overrides for base URLs, API keys, and custom models.

The configuration shape allows you to:

• Override the base URL (baseUrl) for proxy gateways or self-hosted deployments
• Supply API keys (apiKey) per provider
• Enable or disable providers (enabled)
• Add custom models (customModels) or replace built-in definitions (replaceBuiltInModels)

When resolving a provider at runtime, the getProviderConfig function (lines 1558-1564 in providers.ts) checks the static PROVIDERS table first, then falls back to the persisted configuration:

function getProviderConfig(providerId: ProviderId): ProviderConfig | null {
  if (PROVIDERS[providerId]) return PROVIDERS[providerId];
  if (typeof window !== 'undefined') {
    const stored = localStorage.getItem('providersConfig');
    if (stored) {
      const cfg = JSON.parse(stored);
      const p = cfg[providerId];
      if (p) return { ...p, id: providerId };
    }
  }
  return null;
}

LLM Wrapper in lib/ai/llm.ts

The LLM Wrapper instantiates the appropriate SDK client based on the stored configuration. When callLLM or streamLLM executes, it retrieves the provider configuration and creates the corresponding SDK instance.

According to lines 2020-2034 in llm.ts, the wrapper switches on cfg.type to initialize clients:

export async function callLLM(
  providerId: ProviderId,
  modelId: string,
  input: LLMInput,
  thinkingConfig: ThinkingConfig,
) {
  const cfg = getProviderConfig(providerId);
  if (!cfg) throw new Error(`Unknown provider ${providerId}`);

  const client = (() => {
    switch (cfg.type) {
      case 'openai':    return createOpenAI({ apiKey: cfg.apiKey, baseURL: cfg.baseUrl });
      case 'anthropic': return createAnthropic({ apiKey: cfg.apiKey, baseURL: cfg.baseUrl });
      // …other SDK creators
    }
  })();

  const body = getCompatThinkingBodyParams(providerId, modelId, thinkingConfig);
  return client.chat.completions.create({ model: modelId, messages: input.messages, ...body });
}

Configuring Providers Step-by-Step

You can configure LLM providers through the Settings UI or programmatically via the store API.

Using the Settings UI

1. Navigate to Account → Settings in the OpenMAIC interface
2. Select a provider from the dropdown (e.g., openai, anthropic, qwen, deepseek)
3. Enter your API key if the provider requires authentication
4. Toggle Custom URL and supply a baseUrl if using a proxy or self-hosted endpoint
5. Click Add Model to include custom models not in the built-in registry
6. Save changes to persist the configuration to localStorage and sync to the server KV store

The Zustand persist middleware automatically serializes the providersConfig object and syncs it across devices.

Programmatic Configuration

For dynamic provider switching in React components, use the useSettingsStore hook:

import { useSettingsStore } from '@/lib/store/settings';

// Switch to Qwen with a custom gateway
useSettingsStore.setState(state => ({
  providerId: 'qwen',
  modelId: 'qwen3.5-flash',
  providersConfig: {
    ...state.providersConfig,
    qwen: {
      ...state.providersConfig.qwen,
      baseUrl: 'https://my-qwen-proxy.example.com/v1',
      apiKey: 'sk-my-qwen-key',
    },
  },
}));

Adding Custom Providers and Endpoints

OpenMAIC supports adding entirely new providers that follow the OpenAI-compatible REST contract, even if they are not defined in the static PROVIDERS registry.

Injecting a Custom Provider via localStorage

You can add a custom provider by directly manipulating the providersConfig in localStorage:

localStorage.setItem(
  'providersConfig',
  JSON.stringify({
    ...JSON.parse(localStorage.getItem('providersConfig') ?? '{}'),
    mycustom: {
      id: 'mycustom',
      name: 'My Custom LLM',
      type: 'openai',               // Uses OpenAI SDK
      defaultBaseUrl: '',
      baseUrl: 'https://my.api/v1', // Custom endpoint
      apiKey: 'sk-xxxx',
      requiresApiKey: true,
      models: [{ 
        id: 'gpt-x', 
        name: 'GPT‑X', 
        contextWindow: 1_000_000, 
        outputWindow: 128_000, 
        capabilities: { streaming: true, tools: true } 
      }],
    },
  })
);

After reloading the page, the new provider appears in the Settings UI. This works because getProviderConfig falls back to persisted configuration when a provider ID is not found in the static registry.

Practical Code Examples

Retrieve Current LLM Configuration

Use this React hook to access the active provider and model in your components:

import { useSettingsStore } from '@/lib/store/settings';

export function useCurrentLLM() {
  const { providerId, modelId, providersConfig } = useSettingsStore(
    state => ({
      providerId: state.providerId,
      modelId: state.modelId,
      providersConfig: state.providersConfig,
    })
  );

  const provider = providersConfig[providerId];
  const model = provider.models.find(m => m.id === modelId);
  return { provider, model };
}

Direct LLM Invocation with Custom Provider

Call a model programmatically without UI interaction:

import { callLLM } from '@/lib/ai/llm';
import { defaultThinkingConfig } from '@/lib/ai/thinking-config';

async function generateAnswer(prompt: string) {
  const providerId = 'mycustom';
  const modelId = 'gpt-x';
  const thinkingConfig = defaultThinkingConfig;
  
  const resp = await callLLM(
    providerId, 
    modelId, 
    { messages: [{ role: 'user', content: prompt }] },
    thinkingConfig
  );
  
  return resp.choices[0].message.content;
}

Adding Custom Models in Components

Extend a provider with user-defined models:

function AddCustomModel() {
  const { providerId, providersConfig, setState } = useSettingsStore(state => ({
    providerId: state.providerId,
    providersConfig: state.providersConfig,
    setState: state.setState,
  }));

  const addModel = (id: string, name: string) => {
    const cfg = providersConfig[providerId];
    cfg.customModels = [...(cfg.customModels ?? []), { id, name }];
    setState({ providersConfig: { ...providersConfig, [providerId]: cfg } });
  };
  
  // Render form that calls addModel(...)
}

Summary

• OpenMAIC uses three layers: Static registry (lib/ai/providers.ts), persistent store (lib/store/settings.ts), and runtime wrapper (lib/ai/llm.ts)
• Built-in providers include OpenAI, Anthropic, Google Gemini, Qwen, and DeepSeek, accessible via the Settings UI
• Custom endpoints are supported by overriding baseUrl in the provider configuration for proxy gateways or self-hosted models
• New providers can be added without code changes by injecting configuration into localStorage.providersConfig, provided they use the OpenAI-compatible API format
• Configuration persists automatically via Zustand's middleware, syncing between localStorage and the server-side KV store

Frequently Asked Questions

How do I configure a self-hosted or proxy LLM endpoint?

Override the baseUrl field in the provider configuration. In the Settings UI, enable the "Custom URL" toggle and enter your endpoint address (e.g., https://my-proxy.example.com/v1). Programmatically, update providersConfig[providerId].baseUrl in the store. The system uses this URL when instantiating the OpenAI SDK client in lib/ai/llm.ts.

Can I use multiple LLM providers simultaneously in one session?

Yes. While the active provider for the main chat is determined by the global providerId state, you can call callLLM() with different providerId arguments throughout your application. Each invocation resolves its own configuration via getProviderConfig, allowing you to query OpenAI for one task and Anthropic for another within the same session.

Where are my API keys stored?

API keys are stored in the providersConfig object within localStorage (browser) and the account-scoped KV store (server). The Zustand persist middleware handles synchronization. Keys are never exposed to the client-side bundle hardcoded; they are injected at runtime from the configuration store when creating SDK instances in lib/ai/llm.ts.

Why isn't my custom provider appearing in the dropdown?

Ensure you have set the type field to 'openai' (or another supported SDK type) and that the configuration is properly formatted JSON in localStorage.providersConfig. The provider must be added to the parsed configuration object before the Settings component mounts. If using the programmatic approach, verify that getProviderConfig returns your custom entry by checking for it in the merged configuration returned from the store.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}