# How to Implement a Custom LLM Provider in OpenMAIC’s Provider-Neutral Architecture

> Learn how to implement a custom LLM provider in OpenMAIC's provider-neutral architecture. Create an adapter, export a descriptor, and register it to extend LLM capabilities.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-13

---

**Implementing a custom LLM provider in OpenMAIC requires creating an adapter function that conforms to the `LLMAdapter` type, exporting a provider descriptor with a unique ID, and registering it via `registerProvider()` in the client-side registry at application startup.**

OpenMAIC is an open-source framework that abstracts LLM interactions through a provider-neutral architecture, allowing developers to swap models without modifying business logic. This guide walks through the exact implementation steps based on the THU-MAIC/OpenMAIC source code, from writing the API adapter to registering the provider in the runtime registry.

## Understanding the Three-Layer Architecture

OpenMAIC isolates LLM integration into three distinct layers to ensure **configuration**, **invocation**, and **discovery** remain decoupled.

- **Server-side configuration** ([`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts)): Reads credentials from environment variables and exposes only safe metadata—provider ID and description—to the client. The `LLM_ENV_MAP` object maps env-var prefixes like `MYLLM_*` to provider IDs.

- **Client-side registry** ([`lib/ai/provider-registry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/provider-registry.ts)): Maintains a runtime map of `providerId → adapter`. The `registerProvider()` function populates this map, enabling the rest of the application to call `callLLM()` without knowing which concrete API is used.

- **UI and settings** ([`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts)): Merges the server-provided provider list with custom runtime registrations, automatically populating the Model/Provider selector in the settings interface.

## Step‑by‑Step: Implement a Custom LLM Provider

To add a new LLM, you only need to write an adapter, register it, and optionally expose an environment-variable mapping. No other business logic requires changes.

### Step 1: Create the API Adapter

Create a new file that implements the `LLMAdapter` type. This function handles the raw HTTP request to your LLM endpoint and normalizes the response.

```typescript
// src/lib/ai/my-custom-llm/adapter.ts
import type { LLMAdapter, LLMMessage } from '@/lib/ai/types';

export const callCustomLLM: LLMAdapter = async (
  messages: LLMMessage[],
  { apiKey, baseUrl, model }: { apiKey: string; baseUrl: string; model: string }
) => {
  const resp = await fetch(`${baseUrl}/v1/chat/completions`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({ model, messages }),
  });

  if (!resp.ok) {
    const err = await resp.json();
    throw new Error(`CustomLLM error: ${err.error?.message ?? resp.statusText}`);
  }

  const data = await resp.json();
  return { text: data.choices[0].message.content };
};

```

The adapter must accept an array of messages and a configuration object containing `apiKey`, `baseUrl`, and `model`. It returns an object with a `text` property containing the generated content.

### Step 2: Export a Provider Descriptor

Export a provider object that describes the LLM to the rest of the application. This descriptor includes the stable ID used for lookups, the display name shown in the UI, and the adapter function itself.

```typescript
// src/lib/ai/my-custom-llm/provider.ts
import { callCustomLLM } from './adapter';

export const customLLMProvider = {
  /** Stable identifier used everywhere (including the UI) */
  id: 'my-llm',
  /** Human-readable name – shown in the selector */
  name: 'My Awesome LLM',
  /** Default model list if the provider does not expose a model-list endpoint */
  models: ['gpt-4', 'gpt-3.5-turbo'],
  /** The function that the rest of the app will call */
  callLLM: callCustomLLM,
};

```

### Step 3: Register the Provider in the Runtime Registry

Import your provider descriptor into the central registry file and call `registerProvider()`. This_registration happens once at application startup, making the provider immediately available to all components.

```typescript
// src/lib/ai/provider-registry.ts
import { registerProvider } from '@/lib/ai/registry';
import { customLLMProvider } from './my-custom-llm/provider';

// Register the custom provider so that the UI can discover it
registerProvider(customLLMProvider);

```

The `registerProvider` function adds the object to the internal map that `callLLM` consults when resolving a `providerId`.

### Step 4: Configure Environment Variable Mapping (Optional)

If you want the server to inject credentials automatically, extend the **LLM Env-Map** in the server configuration file. This maps environment variable prefixes to your provider ID.

```typescript
// lib/server/provider-config.ts
export const LLM_ENV_MAP = {
  // …existing entries…
  MYLLM: 'my-llm',          // ← maps MYLLM_API_KEY, MYLLM_BASE_URL to the provider ID
};

```

Once added, the server reads `MYLLM_API_KEY` and `MYLLM_BASE_URL` and exposes the provider metadata to the client without transmitting secrets.

### Step 5: Verify the Integration in the UI

OpenMAIC’s settings UI automatically lists every provider that appears in the registry. After restarting the application, navigate to the settings panel and verify that **"My Awesome LLM"** appears in the Model/Provider dropdown. Selecting it stores the choice in the user’s settings, and all subsequent chat completions route through your `callCustomLLM` adapter.

## Complete Minimal Example

Here is a copy-paste ready implementation for a fictional provider:

```typescript
// src/lib/ai/my-llm/adapter.ts
import type { LLMAdapter, LLMMessage } from '@/lib/ai/types';

export const callMyLLM: LLMAdapter = async (
  messages,
  { apiKey, baseUrl, model }
) => {
  const response = await fetch(`${baseUrl}/v1/completions`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({ model, messages }),
  });

  if (!response.ok) {
    const err = await response.json();
    throw new Error(`MyLLM error: ${err.error?.message ?? response.statusText}`);
  }

  const result = await response.json();
  return { text: result.choices[0].message.content };
};

```

```typescript
// src/lib/ai/my-llm/provider.ts
import { callMyLLM } from './adapter';

export const myLLMProvider = {
  id: 'my-llm',
  name: 'MyLLM',
  models: ['my-7b', 'my-13b'],
  callLLM: callMyLLM,
};

```

```typescript
// src/lib/ai/provider-registry.ts
import { registerProvider } from '@/lib/ai/registry';
import { myLLMProvider } from './my-llm/provider';

registerProvider(myLLMProvider);

```

## Summary

- **Adapter**: Implement the `LLMAdapter` type in a new file to handle API communication and response normalization.
- **Descriptor**: Export an object with `id`, `name`, `models`, and `callLLM` properties to define the provider’s metadata.
- **Registration**: Call `registerProvider()` in [`lib/ai/provider-registry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/provider-registry.ts) at startup to add the provider to the runtime map.
- **Server Config**: Optionally update `LLM_ENV_MAP` in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts) to enable automatic credential injection via environment variables.
- **Isolation**: The architecture ensures secrets never leave the server, while the client uses a type-safe registry to invoke the correct adapter by ID alone.

## Frequently Asked Questions

### What is the minimum code required to add a new LLM to OpenMAIC?

You need two files and one registration line: an adapter function that matches the `LLMAdapter` signature, a provider descriptor object exported from a separate file, and a single `registerProvider()` call imported in [`provider-registry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/provider-registry.ts). No changes to UI components or state management are necessary because the registry automatically propagates the new provider to the settings store.

### How does OpenMAIC keep API keys secure when using custom providers?

API keys remain server-side only. The [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts) file reads environment variables and exposes only non-sensitive metadata—such as the provider ID and available models—to the client via the `LLM_ENV_MAP` configuration. The adapter receives credentials only when the server forwards them during the API route handler execution, ensuring keys never appear in client-side bundles.

### Can I register multiple custom providers in the same application?

Yes. The provider registry supports unlimited registrations. Simply create separate adapter files and provider descriptors for each LLM, then call `registerProvider()` for each one in [`lib/ai/provider-registry.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/provider-registry.ts). Each requires a unique `id` string to prevent collisions in the runtime map that `callLLM` consults when routing requests.

### Why does the UI automatically show my custom provider without editing any React components?

The settings UI in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts) merges the list of built-in providers with the runtime registry’s entries. Because `registerProvider()` adds your custom LLM to the same central map that the settings store queries, the selector component receives the updated list through reactive state. This design adheres to the provider-neutrality principle: the UI consumes an abstract list of providers without hardcoding specific implementations.