# Which TTS Backends Support Voice Cloning in huggingface/speech-to-speech and How to Configure Them

> Discover which TTS backends support voice cloning in huggingface/speech-to-speech and learn how to configure them using reference audio or speaker embeddings.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-07

---

**Only Qwen 3 TTS and Pocket TTS support voice cloning in the huggingface/speech-to-speech library, configured via reference audio paths, speaker embeddings, or voice file arguments while Facebook MMS, Kokoro, and ChatTTS provide preset-only synthesis without cloning capabilities.**

The huggingface/speech-to-speech repository offers multiple Text-to-Speech (TTS) backend options, but voice cloning—which synthesizes speech mimicking a reference speaker—is limited to specific handlers. Understanding which TTS backends support voice cloning and how to configure them is essential for building personalized speech-to-speech pipelines.

## TTS Backends That Support Voice Cloning

### Qwen 3 TTS

Qwen 3 TTS provides the most flexible voice cloning implementation in the library, supporting three distinct cloning modes through different input types.

**Configuration parameters** (defined in [`src/speech_to_speech/arguments_classes/qwen3_tts_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/qwen3_tts_arguments.py)):

- `qwen3_tts_ref_audio` – filesystem path to a reference WAV or MP3 file
- `qwen3_tts_ref_spk` – path to a pre-computed speaker embedding file (`.spk` extension)
- `qwen3_tts_ref_rvq` – path to pre-computed acoustic codes (`.rvq` extension)
- `qwen3_tts_xvec_only` – boolean flag enabling x-vector-only cloning mode

The handler implementation in [`src/speech_to_speech/TTS/qwen3_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/qwen3_tts_handler.py) parses these arguments in its `setup` method and internally branches to the appropriate cloning pathway. The x-vector-only mode is specifically recommended for clean starts and language switching scenarios.

### Pocket TTS

Pocket TTS offers cloning through its flexible `voice` parameter, which accepts both preset identifiers and custom audio sources.

**Configuration via the `voice` argument** (handled in [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py)):

- Preset strings: `"alba"`, `"marius"`, or other built-in voice names
- Local audio files: filesystem paths like `"my_voice.wav"`
- Hugging Face repositories: URLs formatted as `"hf://kyutai/tts-voices/custom_voice"`

When `voice` points to an audio file or HF URL, the handler's `setup` method calls `self.model.get_state_for_audio_prompt(voice)` to extract a speaker state, which is then reused for all subsequent generations. This differs from Qwen 3 TTS by not requiring separate embedding pre-computation.

## TTS Backends Without Voice Cloning

The following backends are available but do **not** expose cloning functionality:

| Backend | Cloning Support | Implementation File |
|---------|---------------|---------------------|
| **Facebook MMS TTS** | ❌ No—language-specific VITS synthesis only | [`src/speech_to_speech/TTS/facebookmms_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/facebookmms_handler.py) |
| **Kokoro TTS** | ❌ No—fixed speaker presets only | [`src/speech_to_speech/TTS/kokoro_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/kokoro_handler.py) |
| **ChatTTS** | ❌ No—fixed speaker presets only | [`src/speech_to_speech/TTS/chatTTS_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/chatTTS_handler.py) |

These handlers generate speech solely from text input using their respective models without reference speaker conditioning.

## Configuration Flow Through the Pipeline

All TTS backend selection and parameter passing flows through [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py). The pipeline instantiates handlers based on `RuntimeConfig`, which aggregates arguments from [`src/speech_to_speech/cli.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/cli.py) or direct programmatic configuration.

**Key stages:**

1. **CLI parsing** – [`cli.py`](https://github.com/huggingface/speech-to-speech/blob/main/cli.py) captures user arguments including TTS handler selection and cloning-specific flags
2. **Runtime configuration** – `RuntimeConfig` dataclass stores validated parameters
3. **Handler instantiation** – Pipeline creates the selected handler and calls its `setup` method with configuration arguments
4. **Resource storage** – Cloning-capable handlers persist reference resources (`self.ref_audio`, `self.ref_spk`, `self.voice_state`) for use in `process()`

## Practical Configuration Examples

### Qwen 3 TTS with Raw Reference Audio

```python
from speech_to_speech.pipeline import SpeechToSpeechPipeline
from speech_to_speech.runtime_config import RuntimeConfig

cfg = RuntimeConfig(
    tts_handler="qwen3",
    qwen3_tts_ref_audio="samples/reference.wav",
    qwen3_tts_ref_text="Hello, this is my voice.",
    qwen3_tts_xvec_only=True,  # Recommended for cross-language cloning

)

pipeline = SpeechToSpeechPipeline(cfg)
pipeline.run(...)

```

### Qwen 3 TTS with Pre-computed Speaker Embedding

```python
cfg = RuntimeConfig(
    tts_handler="qwen3",
    qwen3_tts_ref_spk="embeddings/speaker.spk",  # Pre-extracted embedding

    qwen3_tts_xvec_only=True,
)

```

### Pocket TTS with Local Voice File

```python
cfg = RuntimeConfig(
    tts_handler="pocket",
    pocket_tts_voice="recordings/my_voice.wav",
    pocket_tts_device="cuda",
)

```

### Pocket TTS with Hugging Face Voice Repository

```python
cfg = RuntimeConfig(
    tts_handler="pocket",
    pocket_tts_voice="hf://kyutai/tts-voices/custom_speaker",
)

```

## Command-Line Usage

The CLI mirrors programmatic configuration:

**Qwen 3 TTS cloning via CLI:**

```bash
python -m speech_to_speech.cli \
    --tts-handler qwen3 \
    --qwen3-tts-ref-audio samples/reference.wav \
    --qwen3-tts-ref-text "This is the reference transcription." \
    --qwen3-tts-xvec-only

```

**Pocket TTS cloning via CLI:**

```bash
python -m speech_to_speech.cli \
    --tts-handler pocket \
    --pocket-tts-voice my_voice.wav

```

## Summary

- **Qwen 3 TTS** offers the most versatile voice cloning with three input modes (raw audio, speaker embeddings, acoustic codes) plus x-vector-only optimization for language switching
- **Pocket TTS** provides streamlined cloning through its unified `voice` parameter accepting presets, local files, or HF repository URLs
- **Facebook MMS, Kokoro, and ChatTTS** do not support voice cloning and are limited to preset-based synthesis
- All cloning configuration flows through `RuntimeConfig` into handler `setup` methods, with resources stored for reuse during `process()` calls
- Source implementations reside in [`src/speech_to_speech/TTS/qwen3_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/qwen3_tts_handler.py) and [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py)

## Frequently Asked Questions

### What is the difference between Qwen 3 TTS and Pocket TTS voice cloning approaches?

**Qwen 3 TTS** exposes explicit cloning modes with separate parameters for different input types (raw audio, embeddings, acoustic codes) and includes the x-vector-only optimization flag. **Pocket TTS** uses a unified `voice` parameter that auto-detects whether you've provided a preset name, local file path, or HF URL, making it simpler but less configurable for advanced use cases.

### Can I use voice cloning with Facebook MMS TTS or other backends?

No. According to the source code in [`src/speech_to_speech/TTS/facebookmms_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/facebookmms_handler.py), Facebook MMS TTS only synthesizes from text using language-specific VITS models. Similarly, [`src/speech_to_speech/TTS/kokoro_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/kokoro_handler.py) and [`src/speech_to_speech/TTS/chatTTS_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/chatTTS_handler.py) implement fixed-preset synthesis without speaker conditioning. Only Qwen 3 TTS and Pocket TTS currently support cloning.

### What is the x-vector-only mode in Qwen 3 TTS and when should I use it?

The `qwen3_tts_xvec_only` boolean flag enables a cloning mode that uses only speaker x-vectors rather than full acoustic conditioning. According to the implementation, this mode is specifically recommended for clean generation starts and when switching between languages, as it reduces interference from reference audio content while preserving speaker identity characteristics.