# How to Configure LLM Providers and Model Routing in OpenMAIC's lib/ai

> Learn to configure LLM providers and model routing in OpenMAIC's lib/ai. Edit PROVIDERS registry, extend getCompatThinkingBodyParams, and inject custom configurations for flexible AI integration.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-02

---

**Configure LLM providers and model routing in OpenMAIC by editing the `PROVIDERS` registry in [`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts), extending `getCompatThinkingBodyParams` for vendor-specific payloads, and optionally injecting custom providers via `localStorage.providersConfig`.**

The `lib/ai` directory in [OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) implements a provider-agnostic AI layer that unifies multiple LLM vendors behind a consistent interface. This article explains how to add new providers, customize model capabilities, and control request routing using the actual source code architecture.

---

## The Provider Registry Architecture

OpenMAIC centralizes all LLM configuration in a single **provider registry** called `PROVIDERS`. This plain object maps each supported vendor to its connection details, authentication requirements, and available models.

### Step 1: Define Provider Metadata

In [`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts), each provider entry follows this structure:

```typescript
openai: {
  id: 'openai',
  name: 'OpenAI',
  type: 'openai',                    // SDK factory selector
  defaultBaseUrl: 'https://api.openai.com/v1',
  requiresApiKey: true,
  icon: '/logos/openai.svg',
  models: [
    {
      id: 'gpt-4o',
      name: 'GPT-4o',
      contextWindow: 128_000,
      outputWindow: 16_384,
      capabilities: {
        streaming: true,
        tools: true,
        vision: true,
        thinking: { toggleable: true, budgetAdjustable: true, defaultEnabled: false },
      },
    },
    // ... additional models
  ],
},

```

The **`type`** field determines which SDK factory invokes the provider: `createOpenAI`, `createAnthropic`, `createAzure`, `createAmazonBedrock`, or `createGoogleGenerativeAI`.

### Step 2: Enrich with Model Metadata

The last line of [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts) calls `applyModelMetadata(PROVIDERS)`, which enriches each `ModelInfo` with canonical IDs and aliases from [`lib/ai/model-metadata.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/model-metadata.ts). This ensures the UI displays consistent model identifiers even when vendors use varying naming conventions.

---

## Runtime Provider Resolution

The `getProviderConfig(providerId)` function (around line 58 in [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts)) implements a **merge strategy** for runtime configuration:

1. First, it retrieves the built-in provider from `PROVIDERS`
2. Then checks `localStorage.providersConfig` for user-defined overrides
3. Merges custom configurations transparently, allowing endpoint changes or additional models without source code modification

```typescript
const provider = getProviderConfig('openai');
// Returns merged configuration including any localStorage customizations

```

This design enables administrators to deploy OpenMAIC with standard providers while letting end-users add private endpoints or experimental models through the UI.

---

## Model Routing and Request Adaptation

The core of **model routing** happens in `getCompatThinkingBodyParams` (around line 98 in [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts)). This function translates generic `ThinkingConfig` objects into vendor-specific request parameters.

### Understanding Thinking Configuration

The [`thinking-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/thinking-config.ts) file defines helpers for managing reasoning capabilities:

- **`supportsConfigurableThinking`**: Checks if a model supports thinking parameters
- **`pickThinkingEnabled`**: Resolves whether thinking is enabled based on config and defaults
- **`pickThinkingBudget`**: Clamps budget values to model limits
- **`getThinkingConfigKey`**: Generates cache keys for per-model configurations

### Vendor-Specific Payload Mapping

Each provider receives a normalized thinking configuration (`{ mode: 'enabled'|'disabled'|'default', effort?: 'low'|'medium'|'high', budget?: number }`) and returns the appropriate API parameters:

```typescript
// From providers.ts - getCompatThinkingBodyParams implementation
case 'anthropic': {
  const mode = getThinkingMode(config);
  if (mode === 'disabled') return undefined;
  
  return {
    thinking: {
      type: 'enabled',
      budget_tokens: pickThinkingBudget(config, modelInfo),
    },
  };
}

case 'openai': {
  const mode = getThinkingMode(config);
  if (mode === 'disabled') return undefined;
  
  return {
    reasoning_effort: pickThinkingEffort(config), // 'low' | 'medium' | 'high'
  };
}

```

To add support for a new vendor with unique thinking parameters, extend this switch statement with the appropriate transformation.

---

## Adding a Custom LLM Provider

This complete example demonstrates registering a hypothetical provider that uses the OpenAI SDK but with custom thinking semantics.

### Option A: Source Code Modification

Edit [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts) to add the provider to `PROVIDERS`:

```typescript
// In providers.ts, add to the PROVIDERS object
perplexity: {
  id: 'perplexity',
  name: 'Perplexity',
  type: 'openai',
  defaultBaseUrl: 'https://api.perplexity.ai',
  requiresApiKey: true,
  icon: '/logos/perplexity.svg',
  models: [
    {
      id: 'sonar-reasoning-pro',
      name: 'Sonar Reasoning Pro',
      contextWindow: 200_000,
      outputWindow: 8_192,
      capabilities: {
        streaming: true,
        tools: false,
        vision: false,
        thinking: { toggleable: true, budgetAdjustable: false, defaultEnabled: true },
      },
    },
  ],
},

```

Then extend the thinking adaptation:

```typescript
// In getCompatThinkingBodyParams
case 'perplexity': {
  const mode = getThinkingMode(config);
  // Perplexity uses 'search_depth' instead of standard thinking params
  return {
    search_depth: mode === 'enabled' ? 'comprehensive' : 'standard',
  };
}

```

### Option B: Runtime Injection via localStorage

For client-side customization without rebuilding:

```typescript
localStorage.setItem(
  'providersConfig',
  JSON.stringify({
    together: {
      name: 'Together AI',
      type: 'openai',
      defaultBaseUrl: 'https://api.together.xyz/v1',
      requiresApiKey: true,
      icon: '/logos/together.svg',
      models: [
        {
          id: 'meta-llama/Llama-3.3-70B-Instruct-Turbo',
          name: 'Llama 3.3 70B Turbo',
          contextWindow: 131_072,
          outputWindow: 4_096,
          capabilities: {
            streaming: true,
            tools: true,
            vision: false,
            thinking: { toggleable: false, budgetAdjustable: false, defaultEnabled: false },
          },
        },
      ],
    },
  })
);

```

After refreshing, `getProviderConfig('together')` will return this merged configuration, and the UI will automatically list the new model.

---

## Complete Request Flow Example

This runnable pattern shows the full routing pipeline from selection to API call:

```typescript
import { 
  getProviderConfig, 
  findModelById, 
  getCompatThinkingBodyParams,
  type ThinkingConfig 
} from '@/lib/ai/providers';
import { createOpenAI } from '@ai-sdk/openai';

async function routeChatRequest(
  providerId: string,
  modelId: string,
  messages: Array<{ role: string; content: string }>,
  thinkingConfig: ThinkingConfig
) {
  // 1. Resolve provider configuration
  const provider = getProviderConfig(providerId as ProviderId);
  if (!provider) {
    throw new Error(`Provider not found: ${providerId}`);
  }

  // 2. Locate specific model in provider's catalog
  const modelInfo = findModelById(provider.models, modelId);
  if (!modelInfo) {
    throw new Error(`Model ${modelId} not available in ${provider.name}`);
  }

  // 3. Validate thinking compatibility
  if (thinkingConfig.mode !== 'disabled' && !modelInfo.capabilities.thinking?.toggleable) {
    console.warn(`Model ${modelId} does not support thinking; ignoring config`);
  }

  // 4. Instantiate language model client
  const lm = createOpenAI({
    apiKey: process.env[provider.id.toUpperCase() + '_API_KEY'],
    baseUrl: provider.defaultBaseUrl,
  });

  // 5. Build vendor-specific payload parameters
  const thinkingParams = getCompatThinkingBodyParams(
    provider.id as ProviderId,
    modelInfo.id,
    thinkingConfig
  );

  // 6. Execute request with merged configuration
  const response = await lm.chat({
    model: modelInfo.id,
    messages,
    ...thinkingParams,
  });

  return response;
}

```

The `findModelById` helper (also in [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts)) handles canonical ID resolution through the alias system defined in [`lib/ai/model-aliases.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/model-aliases.ts).

---

## Key Files and Responsibilities

| File | Purpose | Critical Functions |
|------|---------|------------------|
| [`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts) | Provider registry, runtime lookup, client creation, routing | `PROVIDERS`, `getProviderConfig`, `createModel`, `getCompatThinkingBodyParams` |
| [`lib/ai/thinking-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/thinking-config.ts) | Thinking parameter normalization | `supportsConfigurableThinking`, `pickThinkingBudget`, `getThinkingConfigKey` |
| [`lib/ai/model-metadata.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/model-metadata.ts) | Model enrichment and canonical IDs | `applyModelMetadata` |
| [`lib/ai/model-aliases.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/model-aliases.ts) | Alternative ID resolution | Alias mapping tables |
| [`lib/ai/azure.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/azure.ts) | Azure OpenAI URL normalization | `normalizeAzureUrl` |

---

## Summary

- The **`PROVIDERS`** object in [`lib/ai/providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/ai/providers.ts) is the single source of truth for all LLM endpoints, authentication requirements, and model capabilities
- **Custom providers** inject through `localStorage.providersConfig` and merge transparently with built-in configurations via `getProviderConfig`
- **Model routing** centralizes in `getCompatThinkingBodyParams`, which transforms generic `ThinkingConfig` into vendor-specific request parameters
- The **type field** selects SDK factories (`createOpenAI`, `createAnthropic`, etc.) while the **capability flags** drive UI availability and request construction
- **Thinking adaptation** extends by adding cases to the provider switch statement in `getCompatThinkingBodyParams`

---

## Frequently Asked Questions

### How do I add a private OpenAI-compatible endpoint to OpenMAIC?

Use the runtime injection method: store a JSON object in `localStorage.providersConfig` with `type: 'openai'`, your `defaultBaseUrl`, and model definitions. `getProviderConfig` automatically merges this with built-in providers. No source modification required.

### What determines which SDK factory creates the language model client?

The `type` field in the provider configuration selects the factory: `openai` uses `createOpenAI`, `anthropic` uses `createAnthropic`, `azure` uses `createAzure`, `bedrock` uses `createAmazonBedrock`, and `google` uses `createGoogleGenerativeAI`. This mapping occurs in the `createModel` helper within [`providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/providers.ts).

### How does OpenMAIC handle models with different "thinking" or reasoning APIs?

`getCompatThinkingBodyParams` normalizes the abstract `ThinkingConfig` interface into vendor-specific parameters. OpenAI receives `reasoning_effort`, Anthropic receives `thinking.budget_tokens`, and other providers can be extended by adding cases to this function's switch statement.

### Can I override individual model parameters without forking the repository?

Yes. The `localStorage.providersConfig` merge strategy allows you to override `defaultBaseUrl`, add models to existing providers, or define entirely new providers. Changes persist client-side and take effect on the next `getProviderConfig` call.