# How OpenMAIC Handles LLM, TTS, and ASR Provider Integrations: A Technical Deep Dive

> Explore OpenMAIC's technical approach to integrating LLM, TTS, and ASR providers. Learn how its unified architecture securely connects external services without exposing credentials.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-10

---

**OpenMAIC uses a unified, data-driven architecture with environment-variable mapping, static registries, and server-side provider implementations to securely integrate LLM, TTS, ASR, and other external services without exposing credentials to the client.**

The THU-MAIC/OpenMAIC repository implements a modular provider system that abstracts away the complexity of connecting to various AI services. This architecture ensures that API keys and sensitive configuration remain server-side while giving operators granular control over which providers are enabled fleet-wide.

## The Three-Layer Provider Architecture

OpenMAIC organizes its provider integrations into distinct layers that separate configuration, metadata, and runtime execution.

### Environment-to-ID Mapping

The system begins with environment variable scanning in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts). This file defines mapping dictionaries—`LLM_ENV_MAP`, `TTS_ENV_MAP`, and `ASR_ENV_MAP`—that translate environment variable prefixes into canonical provider IDs. For example, variables like `OPENAI_API_KEY` or `TTS_LEMONADE_API_KEY` are mapped to internal identifiers such as `openai` or `lemonade`.

### Static Provider Registries

Centralized metadata lives in [`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts), which exports `TTS_PROVIDERS` and `ASR_PROVIDERS` objects. These registries contain static information including display names, required API keys, default base URLs, available models or voices, and supported audio formats. Both client and server code import these constants, ensuring a single source of truth for provider capabilities without leaking sensitive configuration.

### Server-Side Configuration Loading

The `loadEnvSection` function in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts) merges YAML configuration files with environment variables and processes force-disable switches. The `DISABLE_ENV_MAPS` structure extends the regular environment maps to include "keyless" providers like browser-native or ComfyUI integrations, allowing operators to disable capabilities globally using flags such as `ASR_OPENAI_ENABLED=false`.

## How Provider Integration Works at Runtime

The architecture operates through five coordinated stages that move from configuration detection to API execution.

### Stage 1: Environment Variable Resolution

During server startup, the provider-config module scans `process.env` for provider-specific variables. The prefix-to-ID maps translate these into canonical identifiers. This approach allows operators to configure multiple providers for the same capability (e.g., both OpenAI and Azure for TTS) without code changes.

### Stage 2: Registry-Based UI Population

Client-side components import `TTS_PROVIDERS` and `ASR_PROVIDERS` from [`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts) to render provider selection dropdowns. Because the registry includes only public metadata—never API keys—the UI can validate selections and assemble request payloads safely in the browser.

### Stage 3: Force-Disable Enforcement

Operators can globally disable specific providers using `<CAP>_<PREFIX>_ENABLED=false` patterns. When `ASR_OPENAI_ENABLED` is set to `false`, the configuration loader marks `openai-whisper` as unavailable. Client code in [`store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/store/settings.ts) automatically filters disabled providers and triggers fallback logic to select viable alternatives.

### Stage 4: Server-Side API Execution

Actual external API calls happen in dedicated implementation files. [`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts) and [`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts) receive compact configuration objects containing `providerId`, `apiKey`, `baseUrl`, and `modelId`. These modules construct the appropriate HTTP requests—such as POST requests to OpenAI's `/audio/transcriptions` endpoint for Whisper or `/audio/speech` for TTS—and return normalized responses.

### Stage 5: Client State Synchronization

The global store in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts) tracks selected provider IDs (`ttsProviderId`, `asrProviderId`, `llmProviderId`) and validates them against registries using `hasProviderId`. When a provider becomes unavailable due to configuration changes, the sync logic automatically migrates to an enabled alternative, as verified in [`tests/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/settings-server-sync.test.ts).

## Implementation Examples

### Selecting a TTS Provider on the Client

Client components use the registry and global store to build provider requests without accessing credentials:

```typescript
import { TTS_PROVIDERS } from '@/lib/audio/constants';
import { useStore } from '@/lib/store/settings';

// Get the current provider ID from the global store
const { ttsProviderId } = useStore(state => ({
  ttsProviderId: state.ttsProviderId,
}));

// Resolve the full provider config (used to build the request payload)
const provider = TTS_PROVIDERS[ttsProviderId as keyof typeof TTS_PROVIDERS];

// Example: send a request to the server-side TTS route
await fetch('/api/generate/tts', {
  method: 'POST',
  body: JSON.stringify({
    text: 'Hello world',
    providerId: provider.id,
    modelId: provider.defaultModelId,
  }),
});

```

### Processing ASR Requests on the Server

Server API routes in [`app/api/transcription/route.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/app/api/transcription/route.ts) resolve credentials and dispatch to provider implementations:

```typescript
import { ASR_PROVIDERS } from '@/lib/audio/constants';
import { resolveASRBaseUrl, resolveASRApiKey } from '@/lib/server/provider-config';

// Assume the client sent `asrProviderId = 'openai-whisper'`
const providerId = 'openai-whisper';
const config = ASR_PROVIDERS[providerId as keyof typeof ASR_PROVIDERS];

// Resolve credentials and endpoint from env/YAML
const apiKey = resolveASRApiKey(providerId);
const baseUrl = resolveASRBaseUrl(providerId, config.defaultBaseUrl);

// Build a multipart/form-data request for OpenAI Whisper
const form = new FormData();
form.append('file', audioBlob);
form.append('model', config.defaultModelId);

await fetch(`${baseUrl}/audio/transcriptions`, {
  method: 'POST',
  headers: { Authorization: `Bearer ${apiKey}` },
  body: form,
});

```

### Disabling Providers via Environment Variables

Operators can force-disable specific backends without deploying code changes:

```bash

# In the deployment environment

export ASR_OPENAI_ENABLED=false   # disables the OpenAI Whisper backend fleet-wide

export TTS_AZURE_ENABLED=false    # disables Azure TTS

```

The `provider-config` loader processes these flags through `DISABLE_ENV_MAPS`, marking affected providers as unavailable across the entire deployment.

## Key Files in the Provider System

- **[`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts)**: Core configuration loader defining environment maps and disable logic
- **[`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts)**: Static `TTS_PROVIDERS` and `ASR_PROVIDERS` registries
- **[`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts)**: Server-side TTS implementation for OpenAI, Azure, Qwen, and others
- **[`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts)**: Server-side ASR implementation for Whisper, Qwen, and FunASR
- **[`app/api/generate/tts/route.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/app/api/generate/tts/route.ts)**: Public API endpoint routing TTS requests to provider implementations
- **[`app/api/transcription/route.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/app/api/transcription/route.ts)**: Public API endpoint handling speech-to-text requests
- **[`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts)**: Global state management for selected provider IDs and fallback logic
- **[`lib/hooks/use-audio-recorder.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/hooks/use-audio-recorder.ts)**: Client-side hook for ASR provider selection and request building

## Summary

- OpenMAIC implements a **unified provider architecture** that separates environment configuration from runtime execution
- **Environment-variable mapping** in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts) translates deployment settings into canonical provider IDs
- **Static registries** in [`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts) provide both client and server with provider metadata while keeping secrets server-side
- **Force-disable logic** allows operators to turn off specific providers fleet-wide using `*_ENABLED=false` flags
- **Server-side implementations** handle all external API calls, ensuring API keys never reach the client
- **Automatic fallback** in the settings store ensures continuous operation when providers become unavailable

## Frequently Asked Questions

### How does OpenMAIC prevent API keys from leaking to the client?

All credential resolution happens server-side in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts) through functions like `resolveASRApiKey` and `resolveASRBaseUrl`. The client only receives public metadata from `TTS_PROVIDERS` and `ASR_PROVIDERS` in [`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts), which contain display names and supported models but never API keys or base URLs.

### Can I use multiple providers for the same capability simultaneously?

Yes. The environment mapping system supports multiple providers per capability. You can configure both `OPENAI_API_KEY` and `AZURE_TTS_API_KEY` simultaneously. The client can then select between them, and the server will route requests to the appropriate implementation based on the `providerId` sent in the request payload.

### What happens if a configured provider becomes unavailable?

The global store in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts) implements validation logic using `hasProviderId` to check provider availability against the current registry. If a selected provider is disabled via environment variables or removed from configuration, the system automatically falls back to an available alternative, as tested in [`settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/settings-server-sync.test.ts).

### How do I add a new TTS or ASR provider to OpenMAIC?

To add a new provider, you must update three components: add the environment variable mapping to the appropriate `*_ENV_MAP` in [`lib/server/provider-config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/provider-config.ts), add the provider metadata to `TTS_PROVIDERS` or `ASR_PROVIDERS` in [`lib/audio/constants.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/constants.ts), and implement the API-specific logic in [`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts) or [`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts). Finally, expose any new capabilities through the API routes in [`app/api/generate/tts/route.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/app/api/generate/tts/route.ts) or [`app/api/transcription/route.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/app/api/transcription/route.ts).