# How to Use Custom Voice Files with Pocket TTS for Voice Cloning

> Learn to voice clone with custom audio files using Pocket TTS. Easily integrate your own voice data into the Hugging Face speech-to-speech model. Get started now.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-08

---

**To use custom voice files with Pocket TTS for voice cloning, pass a local audio file path, Hugging Face repository URL, or preset name to the `pocket_tts_voice` argument when configuring the `PocketTTSHandler` class.**

The `huggingface/speech-to-speech` pipeline integrates the Kyutai Labs Pocket TTS model through a dedicated handler that supports voice cloning from arbitrary audio samples. By specifying a voice identifier in the handler arguments, you can generate real-time speech that mimics custom speakers while maintaining the pipeline's streaming architecture.

## Understanding the Pocket TTS Architecture

The voice cloning capability is implemented across two primary modules that handle model initialization and voice state management.

### Core Handler Implementation

The `PocketTTSHandler` class in [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py) manages the text-to-speech conversion and voice embedding extraction. When the pipeline starts with `module_kwargs.tts = "pocket"`, this handler instantiates the Kyutai Labs model and loads the specified voice state.

The handler automatically resamples the model's native 24 kHz output to your target sample rate (default 16 kHz) and streams audio in configurable blocks (default 512 samples) for real-time playback.

### Configuration Arguments

Voice cloning parameters are defined in [`src/speech_to_speech/arguments_classes/pocket_tts_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/pocket_tts_arguments.py) through the `PocketTTSHandlerArguments` dataclass. The critical field for custom voices is **`pocket_tts_voice`**, which accepts three distinct input types and passes them to the model's `get_state_for_audio_prompt` method.

## Supported Voice Input Formats

The handler accepts three voice identifier formats through the `voice` parameter:

- **Preset name** — Built-in speakers like `"alba"` or `"jean"` (default). These ship with the model and require no external files.
- **Local audio file** — Absolute path to a WAV file (e.g., `"/path/to/my_voice.wav"`). The handler extracts a voice embedding directly from the audio.
- **Hugging Face repository** — Repository path using the `hf://` protocol (e.g., `"hf://kyutai/tts-voices/custom_voice"`). The handler downloads and caches the voice embedding from the specified repo.

## Command-Line Usage Examples

You can specify custom voices directly via the CLI when launching the speech-to-speech pipeline:

```bash

# Use a built-in preset voice

python -m speech_to_speech.main --tts pocket \
    --pocket_tts_voice alba

# Clone from a local audio file

python -m speech_to_speech.main --tts pocket \
    --pocket_tts_voice /home/user/my_voice.wav

# Load a voice from a Hugging Face repository

python -m speech_to_speech.main --tts pocket \
    --pocket_tts_voice "hf://kyutai/tts-voices/custom_voice"

```

## Programmatic Pipeline Integration

For custom implementations, instantiate the handler manually through the pipeline factory in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py):

```python
from speech_to_speech.arguments_classes.pocket_tts_arguments import PocketTTSHandlerArguments
from speech_to_speech.s2s_pipeline import get_tts_handler
from queue import Queue
from threading import Event

# Configure custom voice arguments

pocket_args = PocketTTSHandlerArguments(
    pocket_tts_device="cuda",              # or "cpu", "mps"

    pocket_tts_voice="/path/to/custom_voice.wav",
    pocket_tts_sample_rate=16000,
    pocket_tts_blocksize=512,
    pocket_tts_max_tokens=80,
)

# Initialize pipeline queues and events

stop_event = Event()
lm_response_q = Queue()      # Receives TTSIn items from LLM

audio_out_q = Queue()        # Receives TTSOut chunks for playback

should_listen = Event()      # Signals STT to resume listening

# Build the handler

tts_handler = get_tts_handler(
    module_kwargs=type("M", (), {"tts": "pocket"}),
    stop_event=stop_event,
    lm_response_queue=lm_response_q,
    send_audio_chunks_queue=audio_out_q,
    should_listen=should_listen,
    chat_tts_handler_kwargs=None,
    facebook_mms_tts_handler_kwargs=None,
    pocket_tts_handler_kwargs=pocket_args,
    kokoro_tts_handler_kwargs=None,
    qwen3_tts_handler_kwargs=None,
)

```

Once initialized, calling `tts_handler.process()` streams synthesized audio using your custom voice embedding.

## How Voice Loading Works Internally

Inside [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py), the `setup()` method processes the voice identifier through the following sequence:

```python

# PocketTTSHandler.setup(...)

logger.info(f"Loading voice: {voice}")
self.voice_state = self.model.get_state_for_audio_prompt(voice)

```

The `get_state_for_audio_prompt` method (provided by the underlying `pocket_tts` library) abstracts the loading logic for all three voice source types. Whether you provide a preset string, local file path, or Hugging Face URL, the method returns a compatible voice state that the handler uses for subsequent audio generation.

## Summary

- **Three input formats** are supported for Pocket TTS voice cloning: preset names, local WAV files, and Hugging Face repository paths via the `hf://` protocol.
- **Configuration** occurs through `PocketTTSHandlerArguments` in [`src/speech_to_speech/arguments_classes/pocket_tts_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/pocket_tts_arguments.py), specifically the `pocket_tts_voice` field.
- **Automatic processing** happens in [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py) through `get_state_for_audio_prompt`, which handles embedding extraction and voice state initialization.
- **Real-time streaming** is maintained with automatic resampling from 24 kHz to your target rate and configurable block sizes for low-latency audio playback.

## Frequently Asked Questions

### What audio format should custom voice files use?

The handler accepts standard WAV files for local voice cloning. The `pocket_tts` library internally handles resampling and preprocessing, though using high-quality recordings (clean speech, minimal background noise) produces better cloning results. The specific sample rate of the input file is handled automatically by the model's embedding extractor.

### Can I use GPU acceleration with Pocket TTS voice cloning?

Yes. Set `pocket_tts_device="cuda"` (or `"mps"` for Apple Silicon) in your `PocketTTSHandlerArguments`. The handler moves the model to the specified device during initialization in `setup()`, while voice state extraction via `get_state_for_audio_prompt` also respects the device configuration for accelerated processing.

### How does the handler handle different sample rates?

The Pocket TTS model generates audio at 24 kHz natively. The handler in [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py) automatically resamples output to the `sample_rate` specified in your arguments (default 16 kHz) before streaming chunks to the audio output queue. This ensures compatibility with the pipeline's audio streamer regardless of the model's native rate.

### Where are the preset voices defined?

Preset voices like `"jean"` (the default) and `"alba"` are built into the Kyutai Labs model weights and referenced by the handler's docstring in [`src/speech_to_speech/TTS/pocket_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/pocket_tts_handler.py). These require no external files or downloads; the `get_state_for_audio_prompt` method retrieves their embeddings directly from the model checkpoint.