How to Implement a Custom LLM Provider in OpenMAIC’s Provider-Neutral Architecture

Implementing a custom LLM provider in OpenMAIC requires creating an adapter function that conforms to the LLMAdapter type, exporting a provider descriptor with a unique ID, and registering it via registerProvider() in the client-side registry at application startup.

OpenMAIC is an open-source framework that abstracts LLM interactions through a provider-neutral architecture, allowing developers to swap models without modifying business logic. This guide walks through the exact implementation steps based on the THU-MAIC/OpenMAIC source code, from writing the API adapter to registering the provider in the runtime registry.

Understanding the Three-Layer Architecture

OpenMAIC isolates LLM integration into three distinct layers to ensure configuration, invocation, and discovery remain decoupled.

• Server-side configuration (lib/server/provider-config.ts): Reads credentials from environment variables and exposes only safe metadata—provider ID and description—to the client. The LLM_ENV_MAP object maps env-var prefixes like MYLLM_* to provider IDs.

• Client-side registry (lib/ai/provider-registry.ts): Maintains a runtime map of providerId → adapter. The registerProvider() function populates this map, enabling the rest of the application to call callLLM() without knowing which concrete API is used.

• UI and settings (lib/store/settings.ts): Merges the server-provided provider list with custom runtime registrations, automatically populating the Model/Provider selector in the settings interface.

Step‑by‑Step: Implement a Custom LLM Provider

To add a new LLM, you only need to write an adapter, register it, and optionally expose an environment-variable mapping. No other business logic requires changes.

Step 1: Create the API Adapter

Create a new file that implements the LLMAdapter type. This function handles the raw HTTP request to your LLM endpoint and normalizes the response.

// src/lib/ai/my-custom-llm/adapter.ts
import type { LLMAdapter, LLMMessage } from '@/lib/ai/types';

export const callCustomLLM: LLMAdapter = async (
  messages: LLMMessage[],
  { apiKey, baseUrl, model }: { apiKey: string; baseUrl: string; model: string }
) => {
  const resp = await fetch(`${baseUrl}/v1/chat/completions`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({ model, messages }),
  });

  if (!resp.ok) {
    const err = await resp.json();
    throw new Error(`CustomLLM error: ${err.error?.message ?? resp.statusText}`);
  }

  const data = await resp.json();
  return { text: data.choices[0].message.content };
};

The adapter must accept an array of messages and a configuration object containing apiKey, baseUrl, and model. It returns an object with a text property containing the generated content.

Step 2: Export a Provider Descriptor

Export a provider object that describes the LLM to the rest of the application. This descriptor includes the stable ID used for lookups, the display name shown in the UI, and the adapter function itself.

// src/lib/ai/my-custom-llm/provider.ts
import { callCustomLLM } from './adapter';

export const customLLMProvider = {
  /** Stable identifier used everywhere (including the UI) */
  id: 'my-llm',
  /** Human-readable name – shown in the selector */
  name: 'My Awesome LLM',
  /** Default model list if the provider does not expose a model-list endpoint */
  models: ['gpt-4', 'gpt-3.5-turbo'],
  /** The function that the rest of the app will call */
  callLLM: callCustomLLM,
};

Step 3: Register the Provider in the Runtime Registry

Import your provider descriptor into the central registry file and call registerProvider(). This_registration happens once at application startup, making the provider immediately available to all components.

// src/lib/ai/provider-registry.ts
import { registerProvider } from '@/lib/ai/registry';
import { customLLMProvider } from './my-custom-llm/provider';

// Register the custom provider so that the UI can discover it
registerProvider(customLLMProvider);

The registerProvider function adds the object to the internal map that callLLM consults when resolving a providerId.

Step 4: Configure Environment Variable Mapping (Optional)

If you want the server to inject credentials automatically, extend the LLM Env-Map in the server configuration file. This maps environment variable prefixes to your provider ID.

// lib/server/provider-config.ts
export const LLM_ENV_MAP = {
  // …existing entries…
  MYLLM: 'my-llm',          // ← maps MYLLM_API_KEY, MYLLM_BASE_URL to the provider ID
};

Once added, the server reads MYLLM_API_KEY and MYLLM_BASE_URL and exposes the provider metadata to the client without transmitting secrets.

Step 5: Verify the Integration in the UI

OpenMAIC’s settings UI automatically lists every provider that appears in the registry. After restarting the application, navigate to the settings panel and verify that "My Awesome LLM" appears in the Model/Provider dropdown. Selecting it stores the choice in the user’s settings, and all subsequent chat completions route through your callCustomLLM adapter.

Complete Minimal Example

Here is a copy-paste ready implementation for a fictional provider:

// src/lib/ai/my-llm/adapter.ts
import type { LLMAdapter, LLMMessage } from '@/lib/ai/types';

export const callMyLLM: LLMAdapter = async (
  messages,
  { apiKey, baseUrl, model }
) => {
  const response = await fetch(`${baseUrl}/v1/completions`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({ model, messages }),
  });

  if (!response.ok) {
    const err = await response.json();
    throw new Error(`MyLLM error: ${err.error?.message ?? response.statusText}`);
  }

  const result = await response.json();
  return { text: result.choices[0].message.content };
};

// src/lib/ai/my-llm/provider.ts
import { callMyLLM } from './adapter';

export const myLLMProvider = {
  id: 'my-llm',
  name: 'MyLLM',
  models: ['my-7b', 'my-13b'],
  callLLM: callMyLLM,
};

// src/lib/ai/provider-registry.ts
import { registerProvider } from '@/lib/ai/registry';
import { myLLMProvider } from './my-llm/provider';

registerProvider(myLLMProvider);

Summary

• Adapter: Implement the LLMAdapter type in a new file to handle API communication and response normalization.
• Descriptor: Export an object with id, name, models, and callLLM properties to define the provider’s metadata.
• Registration: Call registerProvider() in lib/ai/provider-registry.ts at startup to add the provider to the runtime map.
• Server Config: Optionally update LLM_ENV_MAP in lib/server/provider-config.ts to enable automatic credential injection via environment variables.
• Isolation: The architecture ensures secrets never leave the server, while the client uses a type-safe registry to invoke the correct adapter by ID alone.

Frequently Asked Questions

What is the minimum code required to add a new LLM to OpenMAIC?

You need two files and one registration line: an adapter function that matches the LLMAdapter signature, a provider descriptor object exported from a separate file, and a single registerProvider() call imported in provider-registry.ts. No changes to UI components or state management are necessary because the registry automatically propagates the new provider to the settings store.

How does OpenMAIC keep API keys secure when using custom providers?

API keys remain server-side only. The lib/server/provider-config.ts file reads environment variables and exposes only non-sensitive metadata—such as the provider ID and available models—to the client via the LLM_ENV_MAP configuration. The adapter receives credentials only when the server forwards them during the API route handler execution, ensuring keys never appear in client-side bundles.

Can I register multiple custom providers in the same application?

Yes. The provider registry supports unlimited registrations. Simply create separate adapter files and provider descriptors for each LLM, then call registerProvider() for each one in lib/ai/provider-registry.ts. Each requires a unique id string to prevent collisions in the runtime map that callLLM consults when routing requests.

Why does the UI automatically show my custom provider without editing any React components?

The settings UI in lib/store/settings.ts merges the list of built-in providers with the runtime registry’s entries. Because registerProvider() adds your custom LLM to the same central map that the settings store queries, the selector component receives the updated list through reactive state. This design adheres to the provider-neutrality principle: the UI consumes an abstract list of providers without hardcoding specific implementations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}