How to Integrate Custom LLM Providers in OpenMAIC: A Complete Developer Guide

OpenMAIC supports custom LLM providers through a unified configuration system that falls back to user-defined settings in localStorage or server-side YAML when a provider ID is not found in the built-in registry.

OpenMAIC provides a flexible AI-provider architecture that allows developers to integrate virtually any LLM service without modifying core source code. Whether you need to connect to a private API endpoint, a self-hosted model, or an emerging provider not yet included in the official release, the lib/ai/providers.ts registry system enables seamless extension through custom configurations. This guide walks you through the complete process of registering and using proprietary LLM services in your OpenMAIC deployment.

How OpenMAIC's Provider Registry Works

The provider resolution system in OpenMAIC follows a hierarchical lookup pattern defined in lib/ai/providers.ts. At runtime, the system queries three potential sources in sequence:

1. Built-in Registry – The PROVIDERS object containing official integrations (OpenAI, Anthropic, Qwen, etc.)
2. Client-Side Configuration – Custom definitions stored in localStorage under the key 'providersConfig' (see lines 58-84 of providers.ts)
3. Server-Side Configuration – Definitions declared in server-providers.yml and injected into the application state

When getProviderConfig() detects an unknown provider ID, it attempts to retrieve a matching ProviderConfig object from these fallback sources. Once loaded, custom providers receive identical treatment to built-in ones: the callLLM function in lib/ai/llm.ts selects the appropriate SDK adapter (such as createOpenAI or createAnthropic) based on the provider.type field and constructs requests using capability-specific helpers like getCompatThinkingBodyParams.

Step-by-Step Integration of Custom LLM Providers

Follow these concrete steps to register a new LLM service in your OpenMAIC instance.

Step 1: Define the Provider Configuration

Create a configuration object that conforms to the ProviderConfig interface (lines 42-47 of lib/ai/providers.ts). The provider ID must begin with custom- to distinguish it from built-in services.

Required fields include:

• type: SDK adapter to use (openai, anthropic, google, bedrock, or azure)
• baseUrl: Endpoint URL for the LLM API
• apiKey: Authentication token (if requiresApiKey is true)
• models: Array of available models with context windows and capability flags

Step 2: Persist the Configuration

For client-side registration, serialize your configuration and store it in the browser's localStorage:

const customConfig = {
  'custom-myai': {
    name: 'MyCustomAI',
    type: 'openai',
    defaultBaseUrl: 'https://api.mycustom.ai/v1',
    requiresApiKey: true,
    icon: '/logos/mycustom.svg',
    models: [{
      id: 'my-model-1',
      name: 'My Model 1',
      contextWindow: 500000,
      outputWindow: 64000,
      capabilities: { streaming: true, tools: true, vision: false }
    }]
  }
};

localStorage.setItem('providersConfig', JSON.stringify(customConfig));

For server-side registration, add the configuration to server-providers.yml:

custom-llama:
  name: "LLaMA-Custom"
  type: openai
  baseUrl: "https://custom.llama.com/api"
  apiKey: "YOUR-API-KEY"
  icon: "/logos/llama.svg"
  models:
    - id: "llama-2-70b"
      name: "LLaMA-2 70B"
      contextWindow: 131072
      outputWindow: 4096
      capabilities:
        streaming: true
        tools: true
        vision: false

Step 3: Verify Store Integration

The useSettingsStore (defined in store/settings.ts) automatically merges custom configurations with the built-in PROVIDERS registry. Verify your provider appears in the Settings UI dropdown, which reads from useSettingsStore.getState().providersConfig.

Step 4: Execute LLM Calls

Invoke the provider using the standard callLLM interface:

import { getProviderConfig } from '@/lib/ai/providers';
import { callLLM } from '@/lib/ai/llm';

async function generateText() {
  const provider = await getProviderConfig('custom-myai');
  if (!provider) throw new Error('Provider not configured');
  
  const response = await callLLM({
    providerId: provider.id,
    modelId: provider.models[0].id,
    prompt: 'Explain quantum entanglement in simple terms.',
    apiKey: process.env.CUSTOM_API_KEY // optional override
  });
  
  return response.text;
}

Custom Provider Configuration Reference

The ProviderConfig interface requires specific fields to ensure compatibility with OpenMAIC's abstraction layer:

• name: Display name for the UI
• type: Determines which SDK adapter lib/ai/llm.ts instantiates (via createOpenAI, createAnthropic, etc.)
• defaultBaseUrl: Primary API endpoint
• requiresApiKey: Boolean flag indicating authentication requirements
• icon: Path to logo asset in public/logos/ (optional)
• models: Array defining available endpoints with contextWindow, outputWindow, and capabilities (streaming, tools, vision)

The system validates these configurations through test suites located in tests/server/provider-config.test.ts and tests/store/settings-server-sync.test.ts, ensuring type safety across client-server boundaries.

Summary

• OpenMAIC uses a hierarchical resolution system in lib/ai/providers.ts that checks built-in providers first, then falls back to custom configurations from localStorage or server-providers.yml.
• Custom provider IDs must use the custom- prefix and conform to the ProviderConfig interface with fields for type, baseUrl, and models.
• The callLLM function in lib/ai/llm.ts treats custom providers identically to built-in ones, selecting SDK adapters based on the type field.
• Client-side configurations persist in localStorage under 'providersConfig', while server-side definitions live in server-providers.yml.
• Visual customization is supported by placing icon assets in public/logos/ and referencing them in the configuration.

Frequently Asked Questions

Can I integrate LLM providers that don't use the OpenAI API format?

Yes. OpenMAIC supports multiple SDK adapters including Anthropic, Google, Bedrock, and Azure through the type field in your ProviderConfig. As long as your custom provider implements one of these compatible API specifications, the callLLM function in lib/ai/llm.ts will automatically select the correct adapter (e.g., createAnthropic for Claude-compatible endpoints) based on your configuration.

Where should I store API keys for custom providers?

For client-side deployments, store sensitive credentials in environment variables or secure vaults, then pass them as the apiKey parameter when calling callLLM(). For server-side configurations defined in server-providers.yml, include the apiKey field directly in the YAML structure. The system reads these values during initialization as validated in tests/server/provider-config.test.ts.

How do I update or remove a custom provider?

To update a provider, overwrite the existing entry in localStorage (client-side) or edit the entry in server-providers.yml (server-side) and reload the application. To remove a provider, delete the corresponding key from the providersConfig object in localStorage or remove the entry from the YAML file. The useSettingsStore will automatically reflect these changes in the UI.

Can I add multiple custom providers simultaneously?

Yes. The providersConfig object supports multiple entries. Simply include additional provider definitions in your localStorage JSON or server-providers.yml file, ensuring each uses a unique custom- prefixed ID. The registry system loads all valid configurations and makes them available through the unified getProviderConfig() interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}