How to Select Different TTS Backends for Speech-to-Speech: A Complete Configuration Guide

You can select different TTS backends in the huggingface/speech-to-speech repository using the qwen3_tts_backend argument for the Qwen-3-TTS handler, choosing between MLX (Apple Silicon auto-selected), GGML (default on other platforms), or Torch for GPU acceleration.

The speech-to-speech library provides a modular TTS architecture that abstracts every text-to-speech engine behind a unified handler interface. This design lets you swap backends—from Apple's MLX framework to GGML quantization or PyTorch—without changing your application code. The selection happens at pipeline initialization and is governed by platform detection combined with explicit user configuration.

Understanding the TTS Handler Architecture

Every TTS backend in the repository inherits from BaseHandler[TTSIn, TTSOut] as defined in src/speech_to_speech/baseHandler.py. This base class enforces a consistent lifecycle: setup() for initialization, process() for audio generation, and cleanup() for resource teardown.

The pipeline (src/speech_to_speech/s2s_pipeline.py) orchestrates the full speech-to-speech flow. When it receives text from the LLM handler, it wraps the content in a TTSIn object—carrying the text, language, and runtime configuration—and forwards it to your configured TTS handler. The handler returns audio chunks via TTSOut typed queues.

For the Qwen-3-TTS handler (src/speech_to_speech/TTS/qwen3_tts_handler.py), backend selection occurs in three layers:

• Platform detection – automatically routes Apple Silicon to MLX
• Explicit backend choice – GGML or Torch for the faster-qwen3-tts path
• Runtime voice overrides – session-level speaker changes regardless of backend

Backend Selection Logic in Qwen-3-TTS

The Qwen3TTSHandler.setup() method implements automatic platform detection at line 51:

self.backend = "mlx" if platform == "darwin" else "faster_qwen3_tts"

On macOS with Apple Silicon, this loads DEFAULT_MLX_MODEL using mlx_audio.tts.utils.load_model (lines 39-44). For all other platforms, it falls back to the faster_qwen3_tts implementation, where you then choose between ggml (default) or torch via the qwen3_tts_backend argument.

The _normalize_faster_backend() method (lines 66-72) validates and normalizes your backend choice. GGML loads GGUF models with configurable quantization, while Torch uses the PyTorch implementation with explicit device placement.

Streaming behavior differs by backend: MLX defaults to 4-sample chunks (DEFAULT_MLX_STREAMING_CHUNK_SIZE), while faster-qwen3-tts uses 8 (DEFAULT_FASTER_STREAMING_CHUNK_SIZE). Override with qwen3_tts_streaming_chunk_size if latency or throughput needs adjustment.

Available TTS Backends

Backend	Selection Method	Best For	Performance Characteristics
MLX	Auto-selected on darwin platform	Apple Silicon Macs	Lowest latency, Metal-optimized, BF16 default
GGML	--qwen3_tts_backend ggml	CPU inference, quantized deployment	Configurable quantization (BF16 default), GGUF format
Torch	--qwen3_tts_backend torch	CUDA GPUs, PyTorch ecosystem	Flexible device/dtype control, cuda or cpu

Configuring Backends via Command Line

The repository exposes TTS configuration through argument dataclasses mapped to CLI flags. Here's how to invoke each backend:

# Default GGML backend (Linux, Intel macOS, or Windows)

speech-to-speech \
  --tts qwen3 \
  --qwen3_tts_backend ggml \
  --qwen3_tts_ggml_quantization BF16

# Torch backend with CUDA acceleration

speech-to-speech \
  --tts qwen3 \
  --qwen3_tts_backend torch \
  --qwen3_tts_device cuda \
  --qwen3_tts_dtype bfloat16

# MLX backend (auto-selected, but verifiable with)

speech-to-speech \
  --tts qwen3
  # No backend flag needed—platform detection handles it

Programmatic Backend Configuration

For Python applications, instantiate Qwen3TTSHandlerArguments from src/speech_to_speech/arguments_classes/qwen3_tts_arguments.py and pass the result to your handler:

from speech_to_speech.arguments_classes.qwen3_tts_arguments import Qwen3TTSHandlerArguments
from speech_to_speech.TTS.qwen3_tts_handler import Qwen3TTSHandler
from speech_to_speech.s2s_pipeline import SpeechToSpeechPipeline

# Configure for Torch backend on GPU

tts_args = Qwen3TTSHandlerArguments(
    qwen3_tts_backend="torch",
    qwen3_tts_device="cuda",
    qwen3_tts_dtype="float16",
    qwen3_tts_streaming_chunk_size=4,
)

handler = Qwen3TTSHandler(**tts_args.__dict__)
pipeline = SpeechToSpeechPipeline(tts_handler=handler)
pipeline.run()

All fields from the arguments dataclass are passed verbatim to the handler constructor and consumed during setup().

Working with Generation Modes Across Backends

The Qwen-3-TTS handler supports three generation modes that work with any backend. The _model_type() method (lines 62-68) inspects model metadata to route requests appropriately:

Voice Clone Mode

Use reference audio to clone a speaker's voice—backend-agnostic:

args = Qwen3TTSHandlerArguments(
    qwen3_tts_backend="ggml",
    qwen3_tts_ref_audio="samples/speaker.wav",
    qwen3_tts_ref_text="This is the transcription of my reference audio.",
)
handler = Qwen3TTSHandler(**args.__dict__)

# _process_voice_clone (lines 71-89) normalizes audio and streams via GGML

Custom Voice Mode

Select from pretrained speakers without reference audio:

args = Qwen3TTSHandlerArguments(
    qwen3_tts_model_name="Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    qwen3_tts_speaker="Aiden",
)
handler = Qwen3TTSHandler(**args.__dict__)

# _process_custom_voice (lines 709-739) generates streaming audio

Voice Design Mode

Create novel voices from textual descriptions:

args = Qwen3TTSHandlerArguments(
    qwen3_tts_voice_design_prompt="A warm, professional female voice with slight British accent",
)

Session-Level Backend Overrides

The handler supports runtime voice switching through _apply_session_voice_override() (lines 94-138). If a realtime response includes a voice field, the handler rewrites self.speaker or self.ref_audio mid-session. This works identically across MLX, GGML, and Torch backends—no reinitialization required.

Alternative TTS Handlers

Beyond Qwen-3-TTS, the repository provides additional handlers following the same pattern:

• PocketTTSHandler (src/speech_to_speech/TTS/pocket_tts_handler.py) – lightweight, edge-optimized option
• KokoroHandler (src/speech_to_speech/TTS/kokoro_handler.py) – Japanese-focused voices
• FacebookMMSHandler (src/speech_to_speech/TTS/facebookmms_handler.py) – multilingual MMS models with GGUF support
• ChatTTSHandler (src/speech_to_speech/TTS/chatTTS_handler.py) – commercial API integration

Each implements BaseHandler and exposes an *Arguments dataclass for consistent configuration.

Summary

• Platform detection automatically selects MLX on Apple Silicon; other platforms use faster-qwen3-tts
• Explicit backend choice between GGML (default, quantized) and Torch (GPU-flexible) via qwen3_tts_backend
• Configuration layer uses dataclasses (Qwen3TTSHandlerArguments) mapped to CLI flags and Python parameters
• Generation modes (voice-clone, custom-voice, voice-design) work identically across all backends
• Runtime overrides allow session-level voice changes without backend restarts

Frequently Asked Questions

How do I force the Torch backend on Apple Silicon?

Pass qwen3_tts_backend="torch" explicitly. The automatic MLX selection only triggers when platform == "darwin" and no backend is specified. Override the default:

Qwen3TTSHandlerArguments(qwen3_tts_backend="torch", qwen3_tts_device="cpu")

What's the performance difference between GGML and Torch?

GGML uses quantized GGUF models with lower memory footprint—ideal for CPU inference. Torch provides finer device control and performs better on NVIDIA GPUs with CUDA. According to the source, GGML defaults to BF16 quantization while Torch accepts any dtype your hardware supports.

Can I switch TTS backends without restarting the pipeline?

Not directly—the backend is locked at handler initialization in setup(). However, you can switch voices at runtime via the session override mechanism. To change backends, instantiate a new handler and rebuild the pipeline, or run multiple pipeline instances with different configurations.

Where are the GGML quantization options defined?

In src/speech_to_speech/arguments_classes/qwen3_tts_arguments.py, the qwen3_tts_ggml_quantization field accepts values like "BF16", "Q4_K_M", or "Q5_K_S". These map to llama.cpp-style quantization types and affect model size and inference speed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}