Which TTS and ASR Providers Are Supported by OpenMAIC? A Complete Technical Guide

OpenMAIC supports eight TTS providers—including OpenAI, Azure, GLM, Qwen, MiniMax, Doubao, ElevenLabs, and browser-native Web Speech—and five ASR providers including OpenAI Whisper, Azure Speech, Qwen ASR, browser-native recognition, and extensible OpenAI-compatible custom endpoints.

OpenMAIC (THU-MAIC/OpenMAIC) provides a unified audio abstraction layer for speech synthesis and recognition. The system maintains a centralized provider registry in lib/audio/constants.ts that defines metadata, endpoints, and authentication requirements for all supported Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) services, enabling runtime switching between cloud APIs and local browser capabilities.

TTS Providers Available in OpenMAIC

The TTS_PROVIDERS object in lib/audio/constants.ts defines eight distinct speech synthesis options. All cloud providers require API keys, while the browser-native option operates client-side without authentication.

Cloud-Based TTS Providers

• OpenAI TTS (openai-tts): Connects to https://api.openai.com/v1 and offers multiple voice models. Requires an OpenAI API key configured in the settings store.

• Azure TTS (azure-tts): Microsoft Speech Services integration using regional endpoints formatted as https://{region}.tts.speech.microsoft.com. Supports standard and neural voices.

• GLM TTS (glm-tts): Zhipu AI integration via https://open.bigmodel.cn/api/paas/v4. Provides Chinese-optimized synthesis capabilities.

• Qwen TTS (qwen-tts): Alibaba Cloud DashScope service at https://dashscope.aliyuncs.com/api/v1. Designed for high-quality Mandarin and multilingual synthesis.

• MiniMax TTS (minimax-tts): Uses https://api.minimaxi.com/v1 with model definitions stored in the MINIMAX_TTS_MODELS constant. Supports various voice personas.

• Doubao TTS (doubao-tts): ByteDance Ark platform integration at https://ark.cn-beijing.aliyuncs.com/api/v1. Optimized for streaming applications.

• ElevenLabs TTS (elevenlabs-tts): High-quality voice cloning and synthesis via https://api.elevenlabs.io/v1. Added through the extensible provider workflow documented in the codebase.

Browser-Native TTS

• Browser-Native (browser-native): Implements the Web Speech API SpeechSynthesis interface. This client-only provider requires no API keys and functions entirely within the browser, making it ideal for offline applications or privacy-sensitive deployments.

ASR Providers Available in OpenMAIC

The ASR_PROVIDERS registry in lib/audio/constants.ts configures four primary recognition services plus support for custom OpenAI-compatible endpoints.

Core ASR Providers

• OpenAI Whisper (openai-whisper): Cloud-based transcription via https://api.openai.com/v1. Supports multiple audio formats and languages with high accuracy.

• Azure ASR (azure-asr): Microsoft Speech-to-Text service using endpoints formatted as https://{region}.api.cognitive.microsoft.com. Provides real-time and batch transcription capabilities.

• Qwen ASR (qwen-asr): Alibaba Cloud DashScope speech recognition at https://dashscope.aliyuncs.com/api/v1. Optimized for Chinese language transcription.

• Browser-Native ASR (browser-native): Implements the Web Speech API SpeechRecognition interface for client-side transcription without network latency or data transmission costs.

Custom OpenAI-Compatible Providers

The system supports arbitrary OpenAI-compatible ASR endpoints through the CUSTOM_ASR_DEFAULT_LANGUAGES constant. Examples referenced in the source include funasr-asr and lemonade-asr, allowing integration with self-hosted or third-party Whisper-compatible services without modifying core provider code.

Working with Audio Providers Programmatically

OpenMAIC exposes factories in lib/audio/tts-providers.ts and lib/audio/asr-providers.ts that route requests to the appropriate implementation based on the configured provider ID.

Listing Configured TTS Providers

To enumerate available providers and their requirements:

import { TTS_PROVIDERS } from '@/lib/audio/constants';

function listTtsProviders() {
  return Object.values(TTS_PROVIDERS).map(p => ({
    id: p.id,
    name: p.name,
    requiresApiKey: p.requiresApiKey,
    voices: p.voices?.length ?? 0,
  }));
}

console.log(listTtsProviders());

This returns metadata for all registered providers, including voice counts and authentication requirements.

Switching the Active TTS Provider

Runtime provider selection uses the Zustand settings store defined in lib/store/settings.ts:

import { useSettingsStore } from '@/lib/store/settings';

function setActiveTtsProvider(providerId: string) {
  const store = useSettingsStore.getState();
  if (!store.ttsProvidersConfig[providerId]) {
    throw new Error(`Unknown TTS provider: ${providerId}`);
  }
  store.setTTSProvider(providerId);
}

// Switch to Azure TTS
setActiveTtsProvider('azure-tts');

Generating Speech

The generateTTS function dispatches to the concrete implementation based on the current settings:

import { generateTTS } from '@/lib/audio/tts-providers';
import { useSettingsStore } from '@/lib/store/settings';

async function synthesizeSpeech(text: string) {
  const { ttsProviderId, ttsVoice, ttsModelId } = useSettingsStore.getState();
  const audio = await generateTTS(
    { providerId: ttsProviderId, voiceId: ttsVoice, modelId: ttsModelId },
    text,
  );
  // Returns { audio: Uint8Array, format: 'mp3' | ... }
  return audio;
}

Transcribing Audio

Similarly, ASR operations use the transcribeAudio factory:

import { transcribeAudio } from '@/lib/audio/asr-providers';
import { useSettingsStore } from '@/lib/store/settings';

async function transcribeCurrentAudio(blob: Blob) {
  const { asrProviderId, asrLanguage } = useSettingsStore.getState();
  const result = await transcribeAudio(
    { providerId: asrProviderId, language: asrLanguage },
    blob,
  );
  console.log('Transcription:', result.text);
}

Provider Architecture and Key Files

The audio subsystem follows a factory pattern that decouples provider metadata from implementation logic:

• lib/audio/constants.ts: Central registry containing TTS_PROVIDERS and ASR_PROVIDERS objects with endpoint URLs, model lists, and voice configurations.

• lib/audio/tts-providers.ts: Factory that maps provider IDs to concrete TTS implementations and handles request dispatching.

• lib/audio/asr-providers.ts: Factory managing ASR provider instantiation and transcription request routing.

• lib/store/settings.ts: Zustand store persisting the active ttsProviderId, asrProviderId, voice selections, and per-provider API keys.

• lib/types/settings.ts: TypeScript definitions for TTSProviderId, ASRProviderId, and configuration interfaces ensuring type safety across the audio stack.

Summary

• OpenMAIC ships with eight TTS providers (OpenAI, Azure, GLM, Qwen, MiniMax, Doubao, ElevenLabs, and browser-native) and five ASR providers (OpenAI Whisper, Azure, Qwen, browser-native, plus custom endpoints).

• All cloud providers require API keys and connect to their respective regional endpoints as defined in lib/audio/constants.ts.

• The browser-native providers for both TTS and ASR require no authentication and process audio entirely client-side using Web Speech APIs.

• Provider selection and configuration are managed through the useSettingsStore Zustand store, with factory functions in tts-providers.ts and asr-providers.ts handling runtime dispatch.

Frequently Asked Questions

How do I add a new custom ASR provider to OpenMAIC?

Add your OpenAI-compatible endpoint to the CUSTOM_ASR_DEFAULT_LANGUAGES constant in lib/audio/constants.ts following the pattern used by funasr-asr and lemonade-asr. The system will automatically recognize the new provider ID and route requests through the standard OpenAI client implementation with your custom base URL.

Can I use OpenMAIC for speech synthesis without API keys?

Yes. Select the browser-native provider for either TTS or ASR. This uses the browser's built-in Web Speech API, requiring no external services, API keys, or network requests for audio processing.

What happens if a cloud provider is unavailable?

The settings store in lib/store/settings.ts manages provider availability states. If a server-side provider is disabled or fails, the system can fall back to alternative providers based on your configuration, as validated in tests/store/settings-server-sync.test.ts. For production deployments, implement retry logic around the generateTTS and transcribeAudio factory functions.

Which provider offers the best quality for Chinese language processing?

For TTS, Qwen TTS (qwen-tts) and GLM TTS (glm-tts) are optimized for Mandarin Chinese through their respective DashScope and BigModel endpoints. For ASR, Qwen ASR (qwen-asr) provides specialized Chinese speech recognition capabilities via the Alibaba Cloud DashScope API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}