Speech-to-Speech TTS Backends Compared: 5 Options for Text-to-Speech Pipeline

The speech-to-speech repository offers five distinct TTS backends—ChatTTS, Facebook MMS, Pocket TTS, Kokoro, and Qwen 3-TTS—each optimized for different hardware platforms, language requirements, and voice customization needs.

Choosing the right TTS backend for your speech-to-speech pipeline depends on your deployment constraints, target languages, and whether you need voice cloning capabilities. This guide examines each backend's architecture, dependencies, and ideal use cases based on the actual source code implementation in the huggingface/speech-to-speech repository.

ChatTTS: Natural English Speech with Speaker Control

ChatTTS delivers highly natural-sounding English speech with granular speaker-style control through its pure-PyTorch implementation.

The backend loads a single ChatTTS.Chat() instance in src/speech_to_speech/TTS/chatTTS_handler.py and processes requests via the infer() method. It supports CUDA, CPU, and MPS (Apple Silicon) devices through standard PyTorch device mapping.

Key characteristics:

• Language support: English only
• Voice cloning: Not supported—each request generates a fresh voice
• Streaming: Supported with configurable chunk sizes
• Dependencies: ChatTTS, torch, librosa

Use ChatTTS for quick demos and chat-style assistants where a single human-like voice suffices and English is the only required language.

Facebook MMS: Multilingual Coverage Without Custom Voices

Facebook MMS leverages Hugging Face transformers to provide broad language support through VITS-based models.

The handler in src/speech_to_speech/TTS/facebookmms_handler.py instantiates models using VitsModel.from_pretrained() with automatic language mapping defined in WHISPER_LANGUAGE_TO_FACEBOOK_LANGUAGE. Audio output is resampled to 16 kHz to match pipeline requirements.

Key characteristics:

• Language support: 20+ languages via facebook/mms-tts-* model checkpoints
• Voice cloning: Not supported—voice is fixed per language
• Streaming: Supported with 16 kHz resampling
• Dependencies: transformers, torch, librosa, soundfile

Choose Facebook MMS when your application requires multilingual output without custom voice requirements.

Pocket TTS: Lightweight Voice Cloning for Edge Devices

Pocket TTS from Kyutai Labs prioritizes low-latency, on-device synthesis with built-in voice cloning capabilities.

The implementation in src/speech_to_speech/TTS/pocket_tts_handler.py loads TTSModel.load_model() and obtains voice states through model.get_state_for_audio_prompt(). The backend automatically resamples from 24 kHz to the pipeline's standard 16 kHz.

Key characteristics:

• Language support: English with 8 preset voices (alba, jean, liu, michael, nicole, remi, ryan, tian)
• Voice cloning: Supported via reference audio or preset voices
• Streaming: Supported with automatic resampling
• Dependencies: pocket_tts, rich, numpy, scipy

Use Pocket TTS for low-latency edge deployment where custom voice identity is required, such as personalized assistants.

Kokoro: Production-Grade Multilingual with Platform Optimization

Kokoro provides high-quality synthesis across 8 languages with intelligent platform detection for optimal performance.

The handler in src/speech_to_speech/TTS/kokoro_handler.py automatically selects between two backends:

• Non-macOS: Native kokoro C++ library for CUDA/CPU (_setup_kokoro)
• macOS: mlx-audio for Apple Silicon GPU acceleration (_setup_mlx)

Key characteristics:

• Language support: 8 languages with configurable voices per language via KOKORO_LANG_DEFAULT_VOICES
• Voice cloning: Supported via pretrained voices
• Streaming: Supported with platform-optimized inference
• Dependencies: kokoro or mlx-audio, soundfile, espeak-ng (system)

Kokoro excels in production environments requiring fine-grained voice control across multiple languages, especially on Apple Silicon hardware.

Qwen 3-TTS: State-of-the-Art Flexibility for Advanced Use Cases

Qwen 3-TTS represents the most capable backend, supporting multiple inference modes and the highest quality natural speech synthesis.

The handler in src/speech_to_speech/TTS/qwen3_tts_handler.py implements sophisticated backend selection:

• Non-macOS: faster_qwen3_tts with GGML or torch backends (_setup_faster)
• macOS: MLX-compatible model variants (_setup_mlx)

Key characteristics:

• Language support: Any language; speaker names are model-specific (default "Aiden")
• Voice cloning: Multiple modes including custom-voice, voice-design, reference-audio cloning, and pre-computed GGML references
• Streaming: Supported with GGML quantization options
• Dependencies: faster_qwen3_tts, mlx-audio (Apple), soundfile, optional espeak-ng

Select Qwen 3-TTS for scenarios demanding maximum naturalness, custom voice creation capabilities, or large-scale deployment with hardware-specific optimization.

Quick Backend Selection Guide

Your Requirement	Recommended Backend
Fast English with custom voice	pocket
20+ languages, no custom voice	facebookMMS
Highest quality English, speaker control	chatTTS
Production multilingual, preset voices	kokoro
State-of-the-art naturalness, voice creation	qwen3

Configuration Examples

Enable any backend through the --tts flag in s2s_pipeline.py:

ChatTTS (English streaming)

python s2s_pipeline.py \
  --tts chatTTS \
  --chat_tts_device cuda \
  --chat_tts_stream true \
  --chat_tts_chunk_size 512

Facebook MMS (French)

python s2s_pipeline.py \
  --tts facebookMMS \
  --facebook_mms_device cuda \
  --tts_language fr

Pocket TTS (custom voice)

python s2s_pipeline.py \
  --tts pocket \
  --pocket_tts_voice jean \
  --pocket_tts_device cpu \
  --pocket_tts_sample_rate 16000

Kokoro (multilingual)

python s2s_pipeline.py \
  --tts kokoro \
  --kokoro_device auto \
  --kokoro_voice bm_fable \
  --kokoro_lang_code b

Qwen 3-TTS (GGML backend)

python s2s_pipeline.py \
  --tts qwen3 \
  --qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
  --qwen3_tts_device cuda \
  --qwen3_tts_backend ggml \
  --qwen3_tts_speaker Aiden \
  --qwen3_tts_non_streaming_mode true

Programmatic Handler Usage

Instantiate handlers directly for custom pipelines or testing:

from speech_to_speech.TTS.chatTTS_handler import ChatTTSHandler
from speech_to_speech.TTS.facebookmms_handler import FacebookMMSTTSHandler
from speech_to_speech.TTS.pocket_tts_handler import PocketTTSHandler
from speech_to_speech.TTS.kokoro_handler import KokoroTTSHandler
from speech_to_speech.TTS.qwen3_tts_handler import Qwen3TTSHandler
from threading import Event

# ChatTTS example

chat_handler = ChatTTSHandler()
chat_handler.setup(should_listen=Event(), device="cuda", stream=True, chunk_size=512)

# Pocket TTS with custom voice

pocket_handler = PocketTTSHandler()
pocket_handler.setup(should_listen=Event(), device="cpu", voice="jean", sample_rate=16000)

# Qwen 3-TTS with GGML quantization

qwen_handler = Qwen3TTSHandler()
qwen_handler.setup(
    should_listen=Event(),
    model_name="Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    device="cuda",
    backend="ggml",
    ggml_quantization="Q4_K_M",
    speaker="Aiden"
)

Summary

• ChatTTS offers natural English synthesis with speaker control but no voice cloning—ideal for English-only chat applications.
• Facebook MMS covers 20+ languages through Hugging Face transformers, sacrificing voice customization for breadth.
• Pocket TTS minimizes latency and enables voice cloning on CPU, perfect for personalized edge deployment.
• Kokoro automatically optimizes for CUDA/CPU or Apple Silicon while delivering production-quality 8-language synthesis.
• Qwen 3-TTS provides the most flexible architecture with multiple inference backends, quantization options, and advanced voice creation modes.

Legacy backends like parler and melo remain in archive/ but are not integrated into the current pipeline.

Frequently Asked Questions

Which TTS backend supports the most languages?

Facebook MMS supports over 20 languages through the facebook/mms-tts-* model family. Kokoro supports 8 languages with higher per-language voice quality. Qwen 3-TTS theoretically supports any language depending on the model checkpoint used.

Can I use voice cloning with any backend?

Only Pocket TTS, Kokoro, and Qwen 3-TTS support voice cloning. ChatTTS and Facebook MMS generate fixed voices per request (ChatTTS creates fresh random voices, Facebook MMS uses language-specific defaults). Qwen 3-TTS offers the most cloning modes: reference audio, voice design, and pre-computed GGML references.

Which backend works best on Apple Silicon?

Kokoro and Qwen 3-TTS automatically detect macOS and switch to mlx-audio for GPU-accelerated inference. ChatTTS and Pocket TTS support MPS through PyTorch. Facebook MMS falls back to CPU on Apple Silicon unless CUDA is available externally.

What dependencies are required for production deployment?

ChatTTS and Facebook MMS require only Python packages. Kokoro needs espeak-ng as a system dependency for phoneme processing. Qwen 3-TTS may optionally use espeak-ng for voice-design features. Pocket TTS has the lightest footprint with no external system libraries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}