How to Configure Kokoro, Pocket TTS, or ChatTTS Backends in the Speech-to-Speech Library

Use the --tts flag to select a Text-to-Speech backend, then pass backend-specific arguments defined in dedicated argument classes that feed into handler implementations.

The Hugging Face speech-to-speech library enables real-time voice-to-voice conversations by chaining speech recognition, language modeling, and text-to-speech synthesis. Configuring the TTS backend—whether Kokoro, Pocket TTS, or ChatTTS—requires understanding how the library maps command-line flags to Python dataclasses and finally to handler objects that run the actual synthesis engines.

Architecture Overview: From CLI Flags to Synthesis Handlers

The configuration flow follows three layers:

1. Argument dataclasses define valid parameters per backend
2. Pipeline orchestration (prepare_all_args) renames and collects these arguments
3. Handler classes receive the configuration and initialize the underlying TTS engines

This design allows the same pipeline to swap backends without code changes—only different CLI flags.

Kokoro Backend Configuration

Kokoro Argument Class

All Kokoro-specific settings live in KokoroTTSHandlerArguments at src/speech_to_speech/arguments_classes/kokoro_tts_arguments.py:

Field	Description	Default
kokoro_model_name	Model identifier for torch.hub loading	"hexgrad/Kokoro-82M"
kokoro_device	Computation device: auto, cuda, cpu, mps	auto
kokoro_voice	Voice pack to use (e.g., af_bella, bm_fable)	af_bella
kokoro_lang_code	Language code prefix: a (American), b (British), etc.	a
kokoro_speed	Speech speed multiplier	1.0
kokoro_blocksize	Audio chunks per inference step	512

Kokoro Handler Implementation

The KokoroHandler in src/speech_to_speech/TTS/kokoro_handler.py performs:

• Model loading via torch.hub.load style interface
• Device placement and dtype selection
• Voice pack resolution and language code application
• Streaming audio generation with configurable blocksize

# Run Kokoro on Apple Silicon with British voice and faster speech

python -m speech_to_speech \
    --tts kokoro \
    --kokoro_device mps \
    --kokoro_voice bm_fable \
    --kokoro_lang_code b \
    --kokoro_speed 1.2

# Programmatic configuration

from speech_to_speech.arguments_classes.kokoro_tts_arguments import KokoroTTSHandlerArguments

kokoro_config = KokoroTTSHandlerArguments(
    kokoro_device="cuda",
    kokoro_voice="af_nicole",
    kokoro_lang_code="a",
    kokoro_speed=1.0,
    kokoro_blocksize=1024,
)

Pocket TTS Backend Configuration

Pocket TTS Argument Class

The PocketTTSHandlerArguments dataclass at src/speech_to_speech/arguments_classes/pocket_tts_arguments.py exposes:

Field	Description	Default
pocket_tts_device	Target device: cuda, cpu, mps	cpu
pocket_tts_voice	Voice identifier or HuggingFace hub path	None (uses default)
pocket_tts_sample_rate	Output audio sample rate in Hz	16000
pocket_tts_blocksize	Samples per audio chunk	512
pocket_tts_max_tokens	Maximum generation tokens per utterance	5000

Pocket TTS Handler Implementation

The PocketTTSHandler in src/speech_to_speech/TTS/pocket_tts_handler.py:

• Loads models via TTSModel.load_model from the pocket_tts library
• Handles device placement with .cuda() or .to("mps") calls
• Manages voice state preparation through get_state_for_audio_prompt
• Performs on-the-fly resampling and yields 16-bit PCM blocks

# Use Pocket TTS on GPU with custom voice from HuggingFace Hub

python -m speech_to_speech \
    --tts pocket \
    --pocket_tts_device cuda \
    --pocket_tts_voice "hf://kyutai/tts-voices/custom" \
    --pocket_tts_blocksize 1024

# Programmatic Pocket TTS setup

from speech_to_speech.arguments_classes.pocket_tts_arguments import PocketTTSHandlerArguments

pocket_config = PocketTTSHandlerArguments(
    pocket_tts_device="cuda",
    pocket_tts_voice="alba",
    pocket_tts_sample_rate=24000,
    pocket_tts_blocksize=512,
)

ChatTTS Backend Configuration

ChatTTS Argument Class

ChatTTS settings are controlled by ChatTTSHandlerArguments at src/speech_to_speech/arguments_classes/chat_tts_arguments.py:

Field	Description	Default
chat_tts_stream	Enable streaming mode for real-time playback	False
chat_tts_device	Execution device (limited to cuda in practice)	cuda
chat_tts_chunk_size	Audio samples per chunk when streaming	512

ChatTTS Handler Implementation

The ChatTTSHandler in src/speech_to_speech/TTS/chatTTS_handler.py:

• Initializes the third-party ChatTTS.Chat() model
• Defaults to CUDA but respects device configuration
• Handles streaming vs. single-pass inference modes
• Resamples output from 24 kHz to 16 kHz for pipeline compatibility

# Enable streaming ChatTTS with larger chunks

python -m speech_to_speech \
    --tts chat \
    --chat_tts_stream true \
    --chat_tts_device cuda \
    --chat_tts_chunk_size 1024

# Programmatic ChatTTS with voice change at runtime

from speech_to_speech.arguments_classes.chat_tts_arguments import ChatTTSHandlerArguments
import ChatTTS

chat_config = ChatTTSHandlerArguments(
    chat_tts_stream=True,
    chat_tts_device="cuda",
    chat_tts_chunk_size=512,
)

# Runtime voice modification (after handler initialization)

new_voice_emb = chat_handler.model.sample_random_speaker()
chat_handler.params_infer_code = ChatTTS.Chat.InferCodeParams(spk_emb=new_voice_emb)

Pipeline Wiring: How Arguments Reach Handlers

The central orchestration happens in src/speech_to_speech/s2s_pipeline.py. The function prepare_all_args performs argument renaming for cleaner handler access:

# From src/speech_to_speech/s2s_pipeline.py

rename_args(kokoro_tts_handler_kwargs, "kokoro")      # kokoro_tts_handler_kwargs → kokoro

rename_args(pocket_tts_handler_kwargs, "pocket_tts")  # pocket_tts_handler_kwargs → pocket_tts  

rename_args(chat_tts_handler_kwargs, "chat_tts")      # chat_tts_handler_kwargs → chat_tts

The _build_handlers function instantiates the correct handler based on module_kwargs.tts:

# Handler selection logic from s2s_pipeline.py

if module_kwargs.tts == "kokoro":
    tts_handler = KokoroHandler(
        stop_event, queue_in=tts_queue_in, queue_out=tts_queue_out,
        setup_kwargs=vars(kokoro_tts_handler_kwargs)
    )
elif module_kwargs.tts == "pocket":
    tts_handler = PocketTTSHandler(
        stop_event, queue_in=tts_queue_in, queue_out=tts_queue_out,
        setup_kwargs=vars(pocket_tts_handler_kwargs)
    )
elif module_kwargs.tts == "chat":
    tts_handler = ChatTTSHandler(
        stop_event, queue_in=tts_queue_in, queue_out=tts_queue_out,
        setup_kwargs=vars(chat_tts_handler_kwargs)
    )

Complete Programmatic Example

For full pipeline control without CLI parsing:

from speech_to_speech.arguments_classes.kokoro_tts_arguments import KokoroTTSHandlerArguments
from speech_to_speech.arguments_classes.pocket_tts_arguments import PocketTTSHandlerArguments
from speech_to_speech.arguments_classes.chat_tts_arguments import ChatTTSHandlerArguments
from speech_to_speech.s2s_pipeline import ParsedArguments, prepare_all_args, s2s_pipeline

# Configure all three backends (only one will be used based on `tts` field)

kokoro_args = KokoroTTSHandlerArguments(
    kokoro_device="cuda",
    kokoro_voice="af_bella",
    kokoro_lang_code="a",
    kokoro_speed=1.0,
)

pocket_args = PocketTTSHandlerArguments(
    pocket_tts_device="cuda",
    pocket_tts_voice="jean",
)

chat_args = ChatTTSHandlerArguments(
    chat_tts_stream=True,
    chat_tts_chunk_size=1024,
)

# Assemble complete arguments

args = ParsedArguments(
    tts="kokoro",  # Select active backend

    kokoro_tts_handler_kwargs=kokoro_args,
    pocket_tts_handler_kwargs=pocket_args,
    chat_tts_handler_kwargs=chat_args,
    # ... other required arguments ...

)

# Launch pipeline

prepare_all_args(**args.__dict__)
pipeline = s2s_pipeline(args)
pipeline.run()

File Reference Map

Purpose	Path
Kokoro arguments	src/speech_to_speech/arguments_classes/kokoro_tts_arguments.py
Kokoro handler	src/speech_to_speech/TTS/kokoro_handler.py
Pocket TTS arguments	src/speech_to_speech/arguments_classes/pocket_tts_arguments.py
Pocket TTS handler	src/speech_to_speech/TTS/pocket_tts_handler.py
ChatTTS arguments	src/speech_to_speech/arguments_classes/chat_tts_arguments.py
ChatTTS handler	src/speech_to_speech/TTS/chatTTS_handler.py
Pipeline orchestration	src/speech_to_speech/s2s_pipeline.py

Summary

• Select backends with --tts kokoro|pocket|chat — the pipeline instantiates the matching handler
• Configure parameters through backend-specific dataclasses that expose device, voice, speed, and streaming options
• Kokoro offers granular voice control with language codes and speed adjustment
• Pocket TTS supports custom voices via HuggingFace Hub paths and flexible sampling rates
• ChatTTS provides streaming mode for lower latency but requires CUDA
• All configurations flow through prepare_all_args and rename_args in s2s_pipeline.py before reaching handler setup methods

Frequently Asked Questions

How do I switch between TTS backends without modifying code?

Pass the --tts flag with your desired backend name: kokoro, pocket, or chat. The pipeline automatically instantiates the correct handler and ignores configuration for unused backends.

Can I run these backends on CPU or Apple Silicon?

Kokoro supports cpu, cuda, and mps (Apple Silicon). Pocket TTS supports cpu, cuda, and mps. ChatTTS currently defaults to cuda and lacks robust CPU/MPS support in the underlying ChatTTS library.

Where are voice options documented for each backend?

Kokoro voices follow the pattern {lang_code}{gender}_{name} (e.g., af_bella, bm_fable). Pocket TTS voices include built-in identifiers like alba or jean, plus any HuggingFace Hub path. ChatTTS generates voices dynamically via sample_random_speaker() or uses default embeddings.

Why does my ChatTTS configuration include streaming options that don't appear for other backends?

ChatTTS natively supports chunked generation, so the handler exposes chat_tts_stream and chat_tts_chunk_size. Kokoro and Pocket TTS always stream internally using blocksize parameters, but ChatTTS requires explicit streaming enablement for comparable latency characteristics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}