How to Integrate Custom LLM Providers with OpenMAIC: A Complete Guide

You can integrate custom LLM providers with OpenMAIC by registering a configuration object with the custom- prefix in localStorage.providersConfig or the settings store, which the provider registry in lib/ai/providers.ts automatically merges with built-in providers at runtime.

OpenMAIC abstracts every Large Language Model behind a unified provider registry, enabling you to connect proprietary APIs or self-hosted endpoints without modifying the core codebase. By leveraging the extensible architecture in the THU-MAIC/OpenMAIC repository, you can register OpenAI-compatible endpoints or entirely custom implementations through a simple JSON configuration.

Understanding the OpenMAIC Provider Architecture

OpenMAIC manages LLM providers through a centralized registry pattern that separates built-in services from user-defined extensions.

The Provider Registry

In lib/ai/providers.ts, the PROVIDERS constant contains all shipped providers (OpenAI, Anthropic, Bedrock, Google, etc.). When the UI requests a model, the getProviderConfig() function first searches this registry. If the requested providerId is not found, it falls back to the JSON stored in localStorage under the key providersConfig.

// lib/ai/providers.ts - Simplified lookup logic
export function getProviderConfig(providerId: string) {
  if (PROVIDERS[providerId]) {
    return PROVIDERS[providerId];
  }
  // Fallback to custom providers from localStorage
  const customConfig = localStorage.getItem('providersConfig');
  const customProviders = customConfig ? JSON.parse(customConfig) : {};
  return customProviders[providerId];
}

Type Safety Requirements

Custom providers must follow strict naming conventions defined in lib/types/provider.ts. The type system enforces that all custom provider IDs use the prefix custom- (e.g., custom-myai). This is implemented via the type definition ProviderId = BuiltInProviderId | \custom-${string}``, ensuring runtime validation and preventing namespace collisions with built-in services.

State Management

The lib/store/settings.ts file contains a Zustand store that manages the providersConfig object. This store persists custom provider definitions across sessions and merges them with the built-in PROVIDERS constant, making custom models immediately selectable in generation dialogs.

Configuring Your Custom LLM Provider

To successfully integrate custom LLM providers with OpenMAIC, your configuration object must include specific fields that define the endpoint behavior, authentication requirements, and available models.

Required Configuration Fields

Every custom provider requires the following structure:

• id: Must start with custom- (e.g., custom-myai)
• type: Currently supports "openai" for OpenAI-compatible endpoints
• defaultBaseUrl: The root URL for your API endpoint
• requiresApiKey: Boolean indicating if the Authorization header needs a bearer token
• models: Array of model objects, each specifying id, name, contextWindow, outputWindow, and capabilities

{
  "id": "custom-myai",
  "name": "My-AI",
  "type": "openai",
  "defaultBaseUrl": "https://api.my-ai.com/v1",
  "requiresApiKey": true,
  "icon": "/logos/myai.svg",
  "models": [
    {
      "id": "my-model-1",
      "name": "My-AI-1",
      "contextWindow": 200000,
      "outputWindow": 60000,
      "capabilities": { 
        "streaming": true, 
        "tools": true, 
        "vision": false 
      }
    }
  ]
}

The capabilities object tells OpenMAIC which features the model supports, affecting how the UI renders options and how callLLM constructs the request payload.

Integration Methods

You can register your custom provider through the Settings UI or programmatically via the settings store.

Via Settings UI

1. Navigate to Settings → LLM Providers
2. Click "Add custom provider"
3. Paste the JSON configuration into the text area
4. Save the configuration

The UI automatically writes this data to localStorage.providersConfig under the hood. The next time OpenMAIC initializes, the provider registry merges this entry with built-in providers.

Programmatic Registration

For automated deployments or plugin development, use the useSettingsStore to inject providers directly:

// src/utils/customProvider.ts
import { useSettingsStore } from '@/lib/store/settings';

export function registerMyCustomProvider() {
  const cfg = useSettingsStore.getState();
  
  cfg.setProvidersConfig({
    ...cfg.providersConfig,
    'custom-myai': {
      id: 'custom-myai',
      name: 'My-AI',
      type: 'openai',
      defaultBaseUrl: 'https://api.my-ai.com/v1',
      requiresApiKey: true,
      icon: '/logos/myai.svg',
      models: [
        {
          id: 'my-model-1',
          name: 'My-AI-1',
          contextWindow: 200000,
          outputWindow: 60000,
          capabilities: { 
            streaming: true, 
            tools: true, 
            vision: false 
          },
        },
      ],
    },
  });
}

Executing registerMyCustomProvider() during application startup makes the provider immediately available without user interaction.

Making Requests to Custom Providers

Once registered, custom models work identically to built-in providers through the unified LLM interface.

Using callLLM

Import callLLM from lib/ai/llm.ts and reference your custom model using the provider:modelId syntax:

import { callLLM } from '@/lib/ai/llm';

const response = await callLLM(
  {
    model: { 
      provider: 'custom-myai', 
      modelId: 'my-model-1' 
    },
    prompt: 'Explain quantum entanglement in simple terms.',
    maxOutputTokens: 1024,
  },
  'scene-generation',
);

The function resolves the provider via getModelProviderId() → normalizeProviderId() → PROVIDERS / localStorage lookup. It then routes the request to https://api.my-ai.com/v1/chat/completions (or your specified defaultBaseUrl).

Request Handling and Thinking Context

When callLLM or streamLLM executes, the request passes through a thinking context wrapper in lib/ai/llm.ts. For OpenAI-compatible custom providers, the fetch wrapper in providers.ts reads the current thinking configuration and injects provider-specific payload parameters (such as thinking or reasoning_effort) before sending the HTTP request. This ensures advanced features work seamlessly with custom endpoints that support them.

Validating Your Integration

To ensure your custom provider configuration correctly overrides the base URL and request handling, implement unit tests using the pattern found in the OpenMAIC test suite.

// tests/settings/custom-provider-baseurl.test.ts
import { fetchProvider } from '@/lib/ai/providers';

test('custom provider baseUrl is used', async () => {
  const { request } = await fetchProvider('custom-123', 'test-model');
  expect(request.url).toBe('https://example.com/v1/chat/completions');
});

This test verifies that the fetchProvider utility correctly constructs requests using the defaultBaseUrl defined in your custom configuration rather than defaulting to OpenAI's endpoint.

Summary

• Custom providers require the custom- prefix enforced by TypeScript definitions in lib/types/provider.ts to avoid ID collisions with built-in services.
• Store configurations in localStorage.providersConfig via the Settings UI or programmatically through useSettingsStore in lib/store/settings.ts.
• Define OpenAI-compatible endpoints by setting type: "openai" and providing the defaultBaseUrl for your API.
• Reference models using provider:modelId syntax when calling callLLM or streamLLM from lib/ai/llm.ts.
• Validate integrations by testing that fetchProvider in lib/ai/providers.ts correctly applies your custom base URLs and request parameters.

Frequently Asked Questions

What naming convention must custom providers follow in OpenMAIC?

All custom provider IDs must begin with the prefix custom- (for example, custom-myendpoint). This requirement is enforced by the type definition ProviderId = BuiltInProviderId | \custom-${string}` in [lib/types/provider.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/provider.ts). The prefix ensures that user-defined providers never conflict with officially supported providers like OpenAI or Anthropic.

Can I use non-OpenAI compatible endpoints with OpenMAIC custom providers?

Currently, OpenMAIC's custom provider system is optimized for OpenAI-compatible APIs where you set type: "openai" and provide a defaultBaseUrl. The request handling logic in lib/ai/providers.ts specifically formats payloads for the OpenAI chat completions schema. For completely non-compatible endpoints, you would need to modify the core fetch logic or use middleware to translate requests.

Where is custom provider data stored in OpenMAIC?

Custom provider configurations persist in the browser's localStorage under the key providersConfig. The Zustand store in lib/store/settings.ts manages this data, merging it with the built-in PROVIDERS constant at runtime. This architecture allows custom providers to survive page refreshes without requiring a backend database.

How does OpenMAIC handle authentication for custom LLM providers?

When requiresApiKey is set to true in your provider configuration, OpenMAIC expects an API key to be provided in the application settings. The provider registry in lib/ai/providers.ts includes this key in the Authorization: Bearer header when making requests to your defaultBaseUrl. The key is stored securely in the settings store alongside the provider configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}