# Thread Safety Considerations for Multiple Pipeline Instances with `--num_pipelines`

> Learn about thread safety for multiple speech-to-speech pipeline instances with --num_pipelines. Understand isolated state and Apple Silicon MLX lock implications for efficient inference.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: internals
- Published: 2026-08-08

---

**When running the Hugging Face speech-to-speech server with `--num_pipelines` greater than 1, each instance operates in its own thread with isolated state, though Apple Silicon systems enforce a global MLX lock that serializes inference and automatically disables live transcription to prevent warning floods.**

The `huggingface/speech-to-speech` repository supports concurrent processing through the `--num_pipelines` argument, which launches independent pipeline instances to handle multiple audio streams simultaneously. Understanding the thread safety model requires examining how the codebase isolates state, manages system resources, and handles platform-specific constraints in the MLX backend.

## Pipeline Isolation Through Deep Copying and Thread Management

Each pipeline instance created by `build_pipeline()` receives its own independent set of handlers and configuration objects, preventing race conditions between threads.

### Independent Handler Instances

In [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py), the `_build_pipeline_unit()` function creates isolated pipeline units by deep-copying handler configurations. For example, VAD handler arguments are duplicated using `deepcopy(vad_handler_kwargs)` to ensure mutations in one pipeline do not affect others. The backend selections are similarly cloned via `copy_for_pipeline()` methods, establishing strict memory boundaries between instances ([source](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py#L70-L78)).

### ThreadManager Coordination

All handlers execute within dedicated threads managed by the `ThreadManager` class in [`src/speech_to_speech/utils/thread_manager.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/utils/thread_manager.py). The manager instantiates each handler on a separate `threading.Thread` and coordinates graceful shutdown through a shared `stop_event`, ensuring clean termination without resource leaks ([source](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/utils/thread_manager.py#L14-L33)).

## The Apple Silicon MLX Global Lock Constraint

On macOS systems with Apple Silicon, thread safety involves an additional hardware-specific constraint that affects performance characteristics.

### Global Serialization of MLX Inference

As documented in the source comments, all MLX inference operations serialize through a global lock implemented in [`utils/mlx_lock.py`](https://github.com/huggingface/speech-to-speech/blob/main/utils/mlx_lock.py). This means that even with multiple pipeline threads, only one MLX inference operation executes at a time across the entire process, effectively creating contention when `--num_pipelines` exceeds 1 ([source comment](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py#L16-L21)).

### Automatic Disabling of Live Transcription

To prevent an overwhelming flood of contention warnings in the logs, the server automatically disables live transcription when detecting multiple pipelines on Darwin (macOS) platforms. This validation occurs in the `run_pipeline_command` function, which checks the platform and pipeline count before initializing the STT handlers ([source](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py#L22-L28)).

## Device Selection and Resource Contention

While pipeline instances maintain separate Python objects, they may still compete for underlying hardware resources depending on device configuration.

### Shared GPU Resources

The global `device` argument applies to each backend only if that backend's configuration accepts a device parameter. When multiple pipelines target the same CUDA or Metal device, they share the underlying GPU compute and memory without additional synchronization primitives. You must ensure the target device has sufficient capacity for concurrent inference workloads ([source](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py#L38-L46)).

## Validating and Launching Multiple Pipelines

The argument parser enforces minimum values and constructs the pipeline list through `ModuleArguments`.

### Argument Validation

The `num_pipelines` parameter, defined in [`src/speech_to_speech/arguments_classes/module_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py), accepts integer values with a minimum of 1. The `run_pipeline_command` function validates this input, raising a `ValueError` if the specified count is less than 1 ([source definition](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py#L85-L92), [validation logic](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py#L13-L16)).

### Practical Usage Example

To launch a server with two isolated pipelines using the Parakeet STT model and MLX language model backend:

```bash
python -m speech_to_speech serve \
    --num_pipelines 2 \
    --stt parakeet-tdt \
    --llm_backend mlx-lm \
    --tts qwen3

```

For programmatic control, instantiate the `ThreadManager` directly:

```python
from speech_to_speech.s2s_pipeline import parse_arguments, build_pipeline
from threading import Event

args = parse_arguments([
    "--num_pipelines", "3",
    "--stt", "parakeet-tdt",
    "--llm_backend", "mlx-lm",
    "--tts", "qwen3"
])
stop_event = Event()
manager = build_pipeline(args, stop_event)

manager.start()  # Launches 3 independent pipeline threads

manager.wait()   # Blocks until completion

```

## Summary

- **State Isolation**: Each pipeline instance operates on deep-copied handler configurations and separate queues, eliminating race conditions at the application level.
- **MLX Limitation**: Apple Silicon systems enforce a global inference lock that serializes MLX operations across all pipelines, making true parallel inference impossible on these devices.
- **Live Transcription Trade-off**: The server automatically disables live transcription on macOS when using multiple pipelines to avoid log flooding from lock contention warnings.
- **Resource Management**: While Python-level state is isolated, GPU devices are shared across pipelines without additional synchronization, requiring careful memory management.
- **Validation**: The `--num_pipelines` argument enforces a minimum value of 1 through `ModuleArguments` validation.

## Frequently Asked Questions

### Is it safe to run multiple pipelines on Apple Silicon?

While thread-safe at the Python level, running multiple pipelines on Apple Silicon is inefficient because MLX inference serializes through a global lock. The code explicitly disables live transcription on macOS when `--num_pipelines` exceeds 1 to prevent warning spam, though the pipelines will still function sequentially rather than in parallel.

### How does the code prevent race conditions between pipeline instances?

The `build_pipeline()` function creates isolation by deep-copying all handler arguments (such as `vad_handler_kwargs`) and using `copy_for_pipeline()` methods on backend configurations. Each pipeline unit receives its own handler objects and queue instances, ensuring no shared mutable state exists between threads.

### Why is live transcription disabled when using multiple pipelines?

Live transcription is disabled on Darwin platforms when multiple pipelines are active because the global MLX lock causes contention that would generate a flood of warnings. This behavior, implemented in `run_pipeline_command`, keeps logs readable by suppressing the feature rather than allowing continuous contention alerts.

### Can I assign different GPU devices to each pipeline instance?

The current implementation applies a global `device` argument to all backends that support device selection, but does not provide per-pipeline device assignment. All pipelines share the same device configuration, meaning they will compete for resources on the specified GPU unless you run separate processes entirely.