Implementing the Backend Registry Pattern for Swappable Pipeline Components

The huggingface/speech-to-speech library implements a data-driven backend registry in src/speech_to_speech/backend_registry.py that enables runtime swapping of STT, LLM, and TTS handlers through immutable BackendSpec metadata and factory functions.

The speech-to-speech framework achieves its modular processing pipeline through a sophisticated registry system that treats components as interchangeable backends. Implementing the backend registry pattern for swappable pipeline components allows the library to decouple CLI argument parsing from handler instantiation, supporting seamless switching between Whisper, ChatTTS, or custom implementations without modifying core orchestration code.

Architecture Overview

The registry centers on five core abstractions defined in src/speech_to_speech/backend_registry.py.

BackendKind serves as a literal type discriminator ("stt", "llm", "tts") that groups related implementations (line 38). BackendSpec encapsulates immutable metadata for each concrete backend, including its name, kind, argument dataclass, factory function, optional config normalizer, required extra packages, and capability flags (lines 80-90).

BackendSelection represents the resolved choice, pairing a BackendSpec with a normalized configuration dictionary ready for handler construction (lines 101-108). The HandlerContext dataclass provides runtime state—including queues, events, cancel scopes, and speculative-turn trackers—to factory functions during instantiation (lines 58-71).

The build_backend_registry() function validates that registry entries match their declared BackendKind and enforces unique naming conventions (lines 49-60). For safe resolution and instantiation, select_backend() performs lookups while create_backend_handler() invokes factories and translates import errors into actionable user messages (lines 62-92).

How the Registry Powers Swappable Components

The system operates through five distinct mechanisms that separate configuration from implementation.

Definition. Backends register via three constants: STT_BACKENDS, LLM_BACKENDS, and TTS_BACKENDS. Each entry provides a dataclass (e.g., WhisperSTTHandlerArguments) and a factory that constructs the concrete handler.

Normalization. The normalize_dataclass_config() utility converts argument dataclasses to plain dictionaries, ensuring handlers receive only relevant keyword arguments. This decouples CLI parsing from implementation details.

Optional Dependencies. When backends require extra PyPI packages, the required_extra field specifies the dependency (e.g., "whisper-mlx"). If create_backend_handler() catches an import error, it raises an informative ImportError directing users to install the specific extra.

Runtime Selection. Users specify backends by name (e.g., "whisper" for STT) via CLI or configuration files. The select_backend() function resolves this to a BackendSelection, which create_backend_handler() uses to instantiate the handler within a HandlerContext.

Extensibility. Adding support for new providers requires only three elements: an argument dataclass, an importable handler class, and a BackendSpec entry. No modifications to the orchestration logic in src/speech_to_speech/s2s_pipeline.py are necessary.

Code Examples

Selecting and Instantiating a Backend

The following pattern demonstrates runtime backend resolution using the STT registry:

from speech_to_speech.backend_registry import (
    STT_BACKENDS,
    select_backend,
    create_backend_handler,
    HandlerContext,
)
from threading import Event
from queue import Queue

# CancelScope and SpeculativeTurnTracker imported from speech_to_speech.pipeline

# User configuration (typically from CLI or JSON)

user_cfg = {
    "backend": "whisper",
    "model_name": "openai/whisper-small",
    "gen_kwargs": {"max_new_tokens": 200},
}

# 1. Select from the appropriate registry

registry = STT_BACKENDS

# 2. Resolve name and normalize configuration

selection = select_backend(registry, user_cfg["backend"], user_cfg)

# 3. Build runtime context with required queues and events

ctx = HandlerContext(
    stop_event=Event(),
    queue_in=Queue(),
    queue_out=Queue(),
    text_output_queue=Queue(),
    should_listen=Event(),
    cancel_scope=CancelScope(),
    speculative_turns=SpeculativeTurnTracker(),
    pipeline_index=0,
    sample_rate=16000,
    enable_live_transcription=False,
    live_transcription_update_interval=0.1,
)

# 4. Instantiate the concrete handler

handler = create_backend_handler(selection, ctx)

This identical flow applies to LLM and TTS stages by substituting LLM_BACKENDS or TTS_BACKENDS.

Adding a New TTS Backend

Registering a custom text-to-speech implementation requires defining arguments, implementing the handler, and appending to the registry:

from dataclasses import dataclass, field
from speech_to_speech.backend_registry import (
    BackendSpec,
    _simple_handler_factory,
    TTS_BACKENDS,
)

# 1. Define argument dataclass in src/speech_to_speech/arguments_classes

@dataclass
class MyCoolTTSArguments:
    api_key: str
    voice: str = "default"
    gen_kwargs: dict = field(default_factory=dict)

# 2. Implement handler in src/speech_to_speech/TTS/my_cool_handler.py

class MyCoolTTSHandler:
    def __init__(self, stop_event, queue_in, queue_out, *, api_key, voice, gen_kwargs):
        self.api_key = api_key
        self.voice = voice
        self.gen_kwargs = gen_kwargs

# 3. Register by appending to TTS_BACKENDS in backend_registry.py

TTS_BACKENDS.append(
    BackendSpec(
        name="my-cool",
        kind="tts",
        config_type=MyCoolTTSArguments,
        create_handler=_simple_handler_factory(
            "speech_to_speech.TTS.my_cool_handler",
            "MyCoolTTSHandler",
            setup_should_listen=True,
            context_kwargs=True,
        ),
        config_prefix="my_cool_tts",
        required_extra="my-cool-tts",
    )
)

After registration, users activate the backend by setting "backend": "my-cool" in their configuration.

Summary

• The backend registry pattern in src/speech_to_speech/backend_registry.py enables plug-and-play architecture for speech-to-speech pipelines through immutable BackendSpec definitions.
• BackendKind literals ("stt", "llm", "tts") organize handlers into logical groups, while HandlerContext provides runtime dependencies to factory functions.
• Configuration normalization decouples CLI argument parsing from handler instantiation, allowing clean separation of concerns.
• The select_backend() and create_backend_handler() functions provide safe lookup and instantiation with informative error messages for missing optional dependencies.
• Extending the pipeline requires only adding a dataclass, handler implementation, and registry entry—no core pipeline modifications needed.

Frequently Asked Questions

How do I add a new backend to the speech-to-speech pipeline?

Create an argument dataclass to define configuration parameters, implement a handler class that accepts HandlerContext components (queues, events), and append a BackendSpec entry to the appropriate registry constant (STT_BACKENDS, LLM_BACKENDS, or TTS_BACKENDS) in src/speech_to_speech/backend_registry.py. The _simple_handler_factory utility simplifies registration for standard handler patterns.

What happens if a required dependency is missing for a backend?

The create_backend_handler() function catches import errors during factory execution and raises an ImportError with a message specifying the missing required_extra package (e.g., "Install the 'whisper-mlx' extra to use this backend"). This occurs when the factory attempts to import modules that are not installed in the current environment.

How does the registry handle different backend types (STT, LLM, TTS)?

The BackendKind literal type (defined at line 38) enforces categorical separation through three distinct registry constants. Each BackendSpec includes a kind field that must match its registry container, validated by build_backend_registry() to prevent misclassification of handlers across pipeline stages.

Can I use the same backend selection logic for all pipeline stages?

Yes. The select_backend() and create_backend_handler() functions are generic across backend kinds. You simply pass the appropriate registry constant (STT_BACKENDS, LLM_BACKENDS, or TTS_BACKENDS) as the first argument to select_backend(), and the system handles the specific instantiation details through the unified HandlerContext interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}