# How to Integrate Custom TTS or ASR Providers with OpenMAIC

> Learn how to integrate custom TTS or ASR providers with OpenMAIC using its flexible prefix-based system. Centralize configuration and manage providers easily.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-03

---

**OpenMAIC supports custom Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) providers through a prefix-based identification system using `custom-tts-*` and `custom-asr-*` namespaces, with configuration management centralized in [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts) and runtime resolution handled by provider-specific modules in `lib/audio/`.**

OpenMAIC is an open-source multimodal AI framework designed for extensible audio processing workflows. Whether connecting to proprietary cloud APIs or self-hosted speech models, the architecture provides standardized extension points for integrating custom TTS and ASR providers without modifying core framework logic. This guide examines the source code implementation in the THU-MAIC/OpenMAIC repository to demonstrate the precise steps for registering and configuring new speech services.

## Understanding the Provider Identifier Convention

OpenMAIC distinguishes custom providers from built-in ones through strict naming conventions defined in [`lib/audio/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/types.ts). **Custom TTS providers** must use the prefix `custom-tts-` (for example, `custom-tts-myhost` or `custom-tts-siliconflow`), while **custom ASR providers** use the `custom-asr-` prefix (such as `custom-asr-siliconflow`).

The type system exports identifier types `TTSProviderId` and `ASRProviderId`, which include these custom prefixes along with built-in options like browser-native providers. Utility functions `isKnownTTSProviderId()` and `isKnownASRProviderId()` validate provider strings against these type definitions, ensuring any ID starting with the appropriate custom prefix is recognized as valid throughout the application. These validation guards are extensively tested in [`tests/audio/tts-provider-guard.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/tts-provider-guard.test.ts) and [`tests/audio/custom-asr.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/custom-asr.test.ts).

## Configuration Storage and Validation

Provider configurations reside in the global settings store located at [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts). The store maintains two primary configuration objects: `ttsProvidersConfig` and `asrProvidersConfig`, which map provider IDs to their respective settings structures.

Each custom provider configuration typically includes:
- `apiKey`: Authentication credentials for the external service
- `baseUrl`: Endpoint URL for API requests  
- `enabled`: Boolean flag to activate or deactivate the provider
- `customVoices` (TTS only): Array of available voice options with `id` and `name` properties

The helper functions `isTTSProviderConfigured()` and `isASRProviderConfigured()` verify that a custom provider has valid required fields before allowing runtime usage. These guards check both the presence of the provider ID in the configuration store and the existence of essential parameters like API keys, preventing requests to improperly initialized endpoints.

## Server-Side Provider Resolution

When OpenMAIC operates in server-side environments, the framework automatically resolves provider configurations from environment variables. The resolution logic in `lib/audio/` reads variables formatted as `TTS_<PROVIDER>_API_KEY` and `ASR_<PROVIDER>_API_KEY`, mapping them to the corresponding custom provider slots.

Functions `resolveServerTTSProviderId()` and `resolveServerASRProviderId()` handle this initialization, respecting the custom-provider prefixes while maintaining fallback logic to browser-native providers when server configurations are missing or disabled. This automatic hydration ensures the system degrades gracefully if a custom provider loses its server-side credentials, as verified in [`tests/settings/settings-server-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/settings/settings-server-sync.test.ts).

## Implementing a Custom Provider Module

To integrate a new speech service, you must implement a provider module that conforms to the existing interface contracts. The framework expects specific request handlers for TTS and ASR operations located in [`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts) and [`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts).

### Step 1: Configure the Provider Programmatically

First, register your custom provider configuration using the settings store methods. The following example demonstrates setting up a custom TTS provider:

```typescript
import { useSettingsStore } from '@/lib/store/settings';

useSettingsStore.getState().setTTSProviderConfig('custom-tts-myhost', {
  apiKey: 'YOUR_API_KEY',
  baseUrl: 'https://my-tts-service.example.com/v1',
  enabled: true,
  customVoices: [{ id: 'voice-1', name: 'Demo Voice' }],
});

```

For ASR providers, use the corresponding configuration method:

```typescript
import { useSettingsStore } from '@/lib/store/settings';

useSettingsStore.getState().setASRProviderConfig('custom-asr-siliconflow', {
  apiKey: 'YOUR_ASR_KEY',
  baseUrl: 'https://siliconflow-asr.example.com/api',
  enabled: true,
});

```

### Step 2: Implement the Request Handler

Create provider logic within `lib/audio/` that exports functions matching the framework's provider interface. Your implementation should read the stored configuration and perform HTTP calls to your external service.

### Step 3: Register in Provider Registries

Add your provider to the central registries `TTS_PROVIDERS` or `ASR_PROVIDERS` exported from [`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts) and [`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts) respectively. This registration ensures the runtime dispatcher can locate your implementation when the corresponding provider ID is selected in the application settings.

## Runtime Integration and Fallback Behavior

During execution, OpenMAIC routes audio requests through provider-specific dispatchers. The function `applyGenerateTtsResultToScenes()` (utilized in workbench synchronization workflows as tested in [`tests/workbench/tts-stage-sync.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/workbench/tts-stage-sync.test.ts)) retrieves the active `ttsProviderId` from settings and forwards synthesis requests to the appropriate implementation module.

For speech recognition, the `transcribeAudio()` function validates the active `asrProviderId` against configuration stores before invoking the custom provider's transcription logic:

```typescript
import { transcribeAudio } from '@/lib/audio/asr-providers';

const audioBlob = await fetchAudioFile(); // Obtain WAV audio data
const result = await transcribeAudio(
  { providerId: 'custom-asr-siliconflow', language: 'en-US' },
  audioBlob,
);

console.log(result.transcript);

```

If a custom provider becomes unavailable or loses configuration (such as a removed API key), the system automatically falls back to browser-native TTS/ASR capabilities. This fallback mechanism is tested in [`tests/audio/provider-enablement.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/provider-enablement.test.ts) and [`tests/audio/disable-switches-selection.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/disable-switches-selection.test.ts), ensuring the UI never references dead endpoints.

## Summary

- OpenMAIC identifies custom providers using strict prefixes (`custom-tts-*` and `custom-asr-*`) defined in [`lib/audio/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/types.ts)
- Configuration storage happens through [`lib/store/settings.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/store/settings.ts) with validation helpers `isTTSProviderConfigured()` and `isASRProviderConfigured()`
- Server-side initialization automatically reads environment variables formatted as `TTS_<PROVIDER>_API_KEY` and maps them to custom provider configs via `resolveServerTTSProviderId()`
- Implementation requires creating conformant modules under `lib/audio/` and registering them in `TTS_PROVIDERS` or `ASR_PROVIDERS` registries
- Runtime dispatching uses `applyGenerateTtsResultToScenes()` for TTS and `transcribeAudio()` for ASR, with automatic fallback to native providers when custom services fail validation

## Frequently Asked Questions

### How does OpenMAIC validate that a custom provider is properly configured before use?

OpenMAIC uses guard functions `isTTSProviderConfigured()` and `isASRProviderConfigured()` to inspect the settings store before executing requests. These utilities verify that the provider ID exists in `ttsProvidersConfig` or `asrProvidersConfig` and that required fields like `apiKey` and `baseUrl` are non-empty strings. The validation logic prevents runtime errors by blocking requests to improperly initialized providers, as demonstrated in [`tests/audio/provider-enablement.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/provider-enablement.test.ts).

### What environment variables are required for server-side custom provider setup?

Server-side resolution expects environment variables following the pattern `TTS_<PROVIDER>_API_KEY` for text-to-speech and `ASR_<PROVIDER>_API_KEY` for speech recognition, where `<PROVIDER>` corresponds to your custom identifier suffix (the portion after `custom-tts-` or `custom-asr-`). The resolution functions in `lib/audio/` automatically hydrate the settings store with these values on startup, eliminating the need for manual configuration in headless deployments.

### Can I extend the provider configuration to include custom fields like authentication headers or model parameters?

Yes, you can extend the configuration schema in [`lib/audio/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/types.ts) to include additional fields specific to your service. The settings store architecture supports arbitrary configuration objects within `ttsProvidersConfig` and `asrProvidersConfig`. When implementing your provider module in [`lib/audio/tts-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/tts-providers.ts) or [`lib/audio/asr-providers.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/audio/asr-providers.ts), you can access these custom fields through the configuration object passed to your request handler functions.

### What happens if my custom TTS provider API becomes unavailable during runtime?

OpenMAIC implements automatic fallback behavior that switches to browser-native TTS capabilities when a custom provider fails configuration validation or returns errors. The fallback is triggered by the provider guard logic tested in [`tests/audio/tts-provider-guard.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/tts-provider-guard.test.ts) and [`tests/audio/provider-enablement.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/audio/provider-enablement.test.ts), ensuring continuous functionality even when third-party services experience outages or credential expiration.