How to Configure LLM Providers and Model Routing in OpenMAIC's lib/ai

Configure LLM providers and model routing in OpenMAIC by editing the PROVIDERS registry in lib/ai/providers.ts, extending getCompatThinkingBodyParams for vendor-specific payloads, and optionally injecting custom providers via localStorage.providersConfig.

The lib/ai directory in OpenMAIC implements a provider-agnostic AI layer that unifies multiple LLM vendors behind a consistent interface. This article explains how to add new providers, customize model capabilities, and control request routing using the actual source code architecture.

---

The Provider Registry Architecture

OpenMAIC centralizes all LLM configuration in a single provider registry called PROVIDERS. This plain object maps each supported vendor to its connection details, authentication requirements, and available models.

Step 1: Define Provider Metadata

In lib/ai/providers.ts, each provider entry follows this structure:

openai: {
  id: 'openai',
  name: 'OpenAI',
  type: 'openai',                    // SDK factory selector
  defaultBaseUrl: 'https://api.openai.com/v1',
  requiresApiKey: true,
  icon: '/logos/openai.svg',
  models: [
    {
      id: 'gpt-4o',
      name: 'GPT-4o',
      contextWindow: 128_000,
      outputWindow: 16_384,
      capabilities: {
        streaming: true,
        tools: true,
        vision: true,
        thinking: { toggleable: true, budgetAdjustable: true, defaultEnabled: false },
      },
    },
    // ... additional models
  ],
},

The type field determines which SDK factory invokes the provider: createOpenAI, createAnthropic, createAzure, createAmazonBedrock, or createGoogleGenerativeAI.

Step 2: Enrich with Model Metadata

The last line of providers.ts calls applyModelMetadata(PROVIDERS), which enriches each ModelInfo with canonical IDs and aliases from lib/ai/model-metadata.ts. This ensures the UI displays consistent model identifiers even when vendors use varying naming conventions.

---

Runtime Provider Resolution

The getProviderConfig(providerId) function (around line 58 in providers.ts) implements a merge strategy for runtime configuration:

1. First, it retrieves the built-in provider from PROVIDERS
2. Then checks localStorage.providersConfig for user-defined overrides
3. Merges custom configurations transparently, allowing endpoint changes or additional models without source code modification

const provider = getProviderConfig('openai');
// Returns merged configuration including any localStorage customizations

This design enables administrators to deploy OpenMAIC with standard providers while letting end-users add private endpoints or experimental models through the UI.

---

Model Routing and Request Adaptation

The core of model routing happens in getCompatThinkingBodyParams (around line 98 in providers.ts). This function translates generic ThinkingConfig objects into vendor-specific request parameters.

Understanding Thinking Configuration

The thinking-config.ts file defines helpers for managing reasoning capabilities:

• supportsConfigurableThinking: Checks if a model supports thinking parameters
• pickThinkingEnabled: Resolves whether thinking is enabled based on config and defaults
• pickThinkingBudget: Clamps budget values to model limits
• getThinkingConfigKey: Generates cache keys for per-model configurations

Vendor-Specific Payload Mapping

Each provider receives a normalized thinking configuration ({ mode: 'enabled'|'disabled'|'default', effort?: 'low'|'medium'|'high', budget?: number }) and returns the appropriate API parameters:

// From providers.ts - getCompatThinkingBodyParams implementation
case 'anthropic': {
  const mode = getThinkingMode(config);
  if (mode === 'disabled') return undefined;
  
  return {
    thinking: {
      type: 'enabled',
      budget_tokens: pickThinkingBudget(config, modelInfo),
    },
  };
}

case 'openai': {
  const mode = getThinkingMode(config);
  if (mode === 'disabled') return undefined;
  
  return {
    reasoning_effort: pickThinkingEffort(config), // 'low' | 'medium' | 'high'
  };
}

To add support for a new vendor with unique thinking parameters, extend this switch statement with the appropriate transformation.

---

Adding a Custom LLM Provider

This complete example demonstrates registering a hypothetical provider that uses the OpenAI SDK but with custom thinking semantics.

Option A: Source Code Modification

Edit providers.ts to add the provider to PROVIDERS:

// In providers.ts, add to the PROVIDERS object
perplexity: {
  id: 'perplexity',
  name: 'Perplexity',
  type: 'openai',
  defaultBaseUrl: 'https://api.perplexity.ai',
  requiresApiKey: true,
  icon: '/logos/perplexity.svg',
  models: [
    {
      id: 'sonar-reasoning-pro',
      name: 'Sonar Reasoning Pro',
      contextWindow: 200_000,
      outputWindow: 8_192,
      capabilities: {
        streaming: true,
        tools: false,
        vision: false,
        thinking: { toggleable: true, budgetAdjustable: false, defaultEnabled: true },
      },
    },
  ],
},

Then extend the thinking adaptation:

// In getCompatThinkingBodyParams
case 'perplexity': {
  const mode = getThinkingMode(config);
  // Perplexity uses 'search_depth' instead of standard thinking params
  return {
    search_depth: mode === 'enabled' ? 'comprehensive' : 'standard',
  };
}

Option B: Runtime Injection via localStorage

For client-side customization without rebuilding:

localStorage.setItem(
  'providersConfig',
  JSON.stringify({
    together: {
      name: 'Together AI',
      type: 'openai',
      defaultBaseUrl: 'https://api.together.xyz/v1',
      requiresApiKey: true,
      icon: '/logos/together.svg',
      models: [
        {
          id: 'meta-llama/Llama-3.3-70B-Instruct-Turbo',
          name: 'Llama 3.3 70B Turbo',
          contextWindow: 131_072,
          outputWindow: 4_096,
          capabilities: {
            streaming: true,
            tools: true,
            vision: false,
            thinking: { toggleable: false, budgetAdjustable: false, defaultEnabled: false },
          },
        },
      ],
    },
  })
);

After refreshing, getProviderConfig('together') will return this merged configuration, and the UI will automatically list the new model.

---

Complete Request Flow Example

This runnable pattern shows the full routing pipeline from selection to API call:

import { 
  getProviderConfig, 
  findModelById, 
  getCompatThinkingBodyParams,
  type ThinkingConfig 
} from '@/lib/ai/providers';
import { createOpenAI } from '@ai-sdk/openai';

async function routeChatRequest(
  providerId: string,
  modelId: string,
  messages: Array<{ role: string; content: string }>,
  thinkingConfig: ThinkingConfig
) {
  // 1. Resolve provider configuration
  const provider = getProviderConfig(providerId as ProviderId);
  if (!provider) {
    throw new Error(`Provider not found: ${providerId}`);
  }

  // 2. Locate specific model in provider's catalog
  const modelInfo = findModelById(provider.models, modelId);
  if (!modelInfo) {
    throw new Error(`Model ${modelId} not available in ${provider.name}`);
  }

  // 3. Validate thinking compatibility
  if (thinkingConfig.mode !== 'disabled' && !modelInfo.capabilities.thinking?.toggleable) {
    console.warn(`Model ${modelId} does not support thinking; ignoring config`);
  }

  // 4. Instantiate language model client
  const lm = createOpenAI({
    apiKey: process.env[provider.id.toUpperCase() + '_API_KEY'],
    baseUrl: provider.defaultBaseUrl,
  });

  // 5. Build vendor-specific payload parameters
  const thinkingParams = getCompatThinkingBodyParams(
    provider.id as ProviderId,
    modelInfo.id,
    thinkingConfig
  );

  // 6. Execute request with merged configuration
  const response = await lm.chat({
    model: modelInfo.id,
    messages,
    ...thinkingParams,
  });

  return response;
}

The findModelById helper (also in providers.ts) handles canonical ID resolution through the alias system defined in lib/ai/model-aliases.ts.

---

Key Files and Responsibilities

File	Purpose	Critical Functions
lib/ai/providers.ts	Provider registry, runtime lookup, client creation, routing	PROVIDERS, getProviderConfig, createModel, getCompatThinkingBodyParams
lib/ai/thinking-config.ts	Thinking parameter normalization	supportsConfigurableThinking, pickThinkingBudget, getThinkingConfigKey
lib/ai/model-metadata.ts	Model enrichment and canonical IDs	applyModelMetadata
lib/ai/model-aliases.ts	Alternative ID resolution	Alias mapping tables
lib/ai/azure.ts	Azure OpenAI URL normalization	normalizeAzureUrl

---

Summary

• The PROVIDERS object in lib/ai/providers.ts is the single source of truth for all LLM endpoints, authentication requirements, and model capabilities
• Custom providers inject through localStorage.providersConfig and merge transparently with built-in configurations via getProviderConfig
• Model routing centralizes in getCompatThinkingBodyParams, which transforms generic ThinkingConfig into vendor-specific request parameters
• The type field selects SDK factories (createOpenAI, createAnthropic, etc.) while the capability flags drive UI availability and request construction
• Thinking adaptation extends by adding cases to the provider switch statement in getCompatThinkingBodyParams

---

Frequently Asked Questions

How do I add a private OpenAI-compatible endpoint to OpenMAIC?

Use the runtime injection method: store a JSON object in localStorage.providersConfig with type: 'openai', your defaultBaseUrl, and model definitions. getProviderConfig automatically merges this with built-in providers. No source modification required.

What determines which SDK factory creates the language model client?

The type field in the provider configuration selects the factory: openai uses createOpenAI, anthropic uses createAnthropic, azure uses createAzure, bedrock uses createAmazonBedrock, and google uses createGoogleGenerativeAI. This mapping occurs in the createModel helper within providers.ts.

How does OpenMAIC handle models with different "thinking" or reasoning APIs?

getCompatThinkingBodyParams normalizes the abstract ThinkingConfig interface into vendor-specific parameters. OpenAI receives reasoning_effort, Anthropic receives thinking.budget_tokens, and other providers can be extended by adding cases to this function's switch statement.

Can I override individual model parameters without forking the repository?

Yes. The localStorage.providersConfig merge strategy allows you to override defaultBaseUrl, add models to existing providers, or define entirely new providers. Changes persist client-side and take effect on the next getProviderConfig call.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}