# Implementing the Backend Registry Pattern for Swappable Pipeline Components

> Learn how the huggingface/speech-to-speech library uses a backend registry pattern for swappable STT, LLM, and TTS components at runtime. Explore its implementation and benefits.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: architecture
- Published: 2026-08-09

---

**The huggingface/speech-to-speech library implements a data-driven backend registry in [`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py) that enables runtime swapping of STT, LLM, and TTS handlers through immutable `BackendSpec` metadata and factory functions.**

The speech-to-speech framework achieves its modular processing pipeline through a sophisticated registry system that treats components as interchangeable backends. Implementing the backend registry pattern for swappable pipeline components allows the library to decouple CLI argument parsing from handler instantiation, supporting seamless switching between Whisper, ChatTTS, or custom implementations without modifying core orchestration code.

## Architecture Overview

The registry centers on five core abstractions defined in [`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py).

**BackendKind** serves as a literal type discriminator (`"stt"`, `"llm"`, `"tts"`) that groups related implementations (line 38). **BackendSpec** encapsulates immutable metadata for each concrete backend, including its name, kind, argument dataclass, factory function, optional config normalizer, required extra packages, and capability flags (lines 80-90).

**BackendSelection** represents the resolved choice, pairing a `BackendSpec` with a normalized configuration dictionary ready for handler construction (lines 101-108). The **HandlerContext** dataclass provides runtime state—including queues, events, cancel scopes, and speculative-turn trackers—to factory functions during instantiation (lines 58-71).

The `build_backend_registry()` function validates that registry entries match their declared `BackendKind` and enforces unique naming conventions (lines 49-60). For safe resolution and instantiation, `select_backend()` performs lookups while `create_backend_handler()` invokes factories and translates import errors into actionable user messages (lines 62-92).

## How the Registry Powers Swappable Components

The system operates through five distinct mechanisms that separate configuration from implementation.

**Definition.** Backends register via three constants: `STT_BACKENDS`, `LLM_BACKENDS`, and `TTS_BACKENDS`. Each entry provides a dataclass (e.g., `WhisperSTTHandlerArguments`) and a factory that constructs the concrete handler.

**Normalization.** The `normalize_dataclass_config()` utility converts argument dataclasses to plain dictionaries, ensuring handlers receive only relevant keyword arguments. This decouples CLI parsing from implementation details.

**Optional Dependencies.** When backends require extra PyPI packages, the `required_extra` field specifies the dependency (e.g., `"whisper-mlx"`). If `create_backend_handler()` catches an import error, it raises an informative `ImportError` directing users to install the specific extra.

**Runtime Selection.** Users specify backends by name (e.g., `"whisper"` for STT) via CLI or configuration files. The `select_backend()` function resolves this to a `BackendSelection`, which `create_backend_handler()` uses to instantiate the handler within a `HandlerContext`.

**Extensibility.** Adding support for new providers requires only three elements: an argument dataclass, an importable handler class, and a `BackendSpec` entry. No modifications to the orchestration logic in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py) are necessary.

## Code Examples

### Selecting and Instantiating a Backend

The following pattern demonstrates runtime backend resolution using the STT registry:

```python
from speech_to_speech.backend_registry import (
    STT_BACKENDS,
    select_backend,
    create_backend_handler,
    HandlerContext,
)
from threading import Event
from queue import Queue

# CancelScope and SpeculativeTurnTracker imported from speech_to_speech.pipeline

# User configuration (typically from CLI or JSON)

user_cfg = {
    "backend": "whisper",
    "model_name": "openai/whisper-small",
    "gen_kwargs": {"max_new_tokens": 200},
}

# 1. Select from the appropriate registry

registry = STT_BACKENDS

# 2. Resolve name and normalize configuration

selection = select_backend(registry, user_cfg["backend"], user_cfg)

# 3. Build runtime context with required queues and events

ctx = HandlerContext(
    stop_event=Event(),
    queue_in=Queue(),
    queue_out=Queue(),
    text_output_queue=Queue(),
    should_listen=Event(),
    cancel_scope=CancelScope(),
    speculative_turns=SpeculativeTurnTracker(),
    pipeline_index=0,
    sample_rate=16000,
    enable_live_transcription=False,
    live_transcription_update_interval=0.1,
)

# 4. Instantiate the concrete handler

handler = create_backend_handler(selection, ctx)

```

This identical flow applies to LLM and TTS stages by substituting `LLM_BACKENDS` or `TTS_BACKENDS`.

### Adding a New TTS Backend

Registering a custom text-to-speech implementation requires defining arguments, implementing the handler, and appending to the registry:

```python
from dataclasses import dataclass, field
from speech_to_speech.backend_registry import (
    BackendSpec,
    _simple_handler_factory,
    TTS_BACKENDS,
)

# 1. Define argument dataclass in src/speech_to_speech/arguments_classes

@dataclass
class MyCoolTTSArguments:
    api_key: str
    voice: str = "default"
    gen_kwargs: dict = field(default_factory=dict)

# 2. Implement handler in src/speech_to_speech/TTS/my_cool_handler.py

class MyCoolTTSHandler:
    def __init__(self, stop_event, queue_in, queue_out, *, api_key, voice, gen_kwargs):
        self.api_key = api_key
        self.voice = voice
        self.gen_kwargs = gen_kwargs

# 3. Register by appending to TTS_BACKENDS in backend_registry.py

TTS_BACKENDS.append(
    BackendSpec(
        name="my-cool",
        kind="tts",
        config_type=MyCoolTTSArguments,
        create_handler=_simple_handler_factory(
            "speech_to_speech.TTS.my_cool_handler",
            "MyCoolTTSHandler",
            setup_should_listen=True,
            context_kwargs=True,
        ),
        config_prefix="my_cool_tts",
        required_extra="my-cool-tts",
    )
)

```

After registration, users activate the backend by setting `"backend": "my-cool"` in their configuration.

## Summary

- The backend registry pattern in [`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py) enables plug-and-play architecture for speech-to-speech pipelines through immutable `BackendSpec` definitions.
- **BackendKind** literals (`"stt"`, `"llm"`, `"tts"`) organize handlers into logical groups, while **HandlerContext** provides runtime dependencies to factory functions.
- Configuration normalization decouples CLI argument parsing from handler instantiation, allowing clean separation of concerns.
- The `select_backend()` and `create_backend_handler()` functions provide safe lookup and instantiation with informative error messages for missing optional dependencies.
- Extending the pipeline requires only adding a dataclass, handler implementation, and registry entry—no core pipeline modifications needed.

## Frequently Asked Questions

### How do I add a new backend to the speech-to-speech pipeline?

Create an argument dataclass to define configuration parameters, implement a handler class that accepts `HandlerContext` components (queues, events), and append a `BackendSpec` entry to the appropriate registry constant (`STT_BACKENDS`, `LLM_BACKENDS`, or `TTS_BACKENDS`) in [`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py). The `_simple_handler_factory` utility simplifies registration for standard handler patterns.

### What happens if a required dependency is missing for a backend?

The `create_backend_handler()` function catches import errors during factory execution and raises an `ImportError` with a message specifying the missing `required_extra` package (e.g., `"Install the 'whisper-mlx' extra to use this backend"`). This occurs when the factory attempts to import modules that are not installed in the current environment.

### How does the registry handle different backend types (STT, LLM, TTS)?

The `BackendKind` literal type (defined at line 38) enforces categorical separation through three distinct registry constants. Each `BackendSpec` includes a `kind` field that must match its registry container, validated by `build_backend_registry()` to prevent misclassification of handlers across pipeline stages.

### Can I use the same backend selection logic for all pipeline stages?

Yes. The `select_backend()` and `create_backend_handler()` functions are generic across backend kinds. You simply pass the appropriate registry constant (`STT_BACKENDS`, `LLM_BACKENDS`, or `TTS_BACKENDS`) as the first argument to `select_backend()`, and the system handles the specific instantiation details through the unified `HandlerContext` interface.