How to Configure Device Placement for STT, LLM, and TTS on Apple Silicon (MPS)

Use --mac-optimal-settings to automatically place all components on MPS, or override specific modules with flags like --stt_device, --llm_device, and --qwen3_tts_device for granular control.

The huggingface/speech-to-speech pipeline supports Apple Silicon's Metal Performance Shaders (MPS) backend for accelerated inference across speech-to-text (STT), large language model (LLM), and text-to-speech (TTS) components. This guide covers the device configuration system as implemented in the source code, including macOS-specific presets, global device overrides, and component-level overrides.

Understanding the MPS Device Configuration System

The pipeline uses a three-tier configuration hierarchy:

1. macOS preset (--mac-optimal-settings) sets MPS defaults for all supported components
2. Global device (--device) overrides all component devices uniformly
3. Component-specific flags target individual modules without affecting others

This system is managed through two key functions in src/speech_to_speech/s2s_pipeline.py: _mac_preset_defaults and overwrite_device_argument.

The macOS Preset: _mac_preset_defaults

The _mac_preset_defaults function defines optimal device settings for Apple Silicon hardware. According to the source code, it assigns MPS as the default device for each component handler:

# src/speech_to_speech/s2s_pipeline.py (lines 92-103)

def _mac_preset_defaults(args):
    # Optimized defaults for Apple Silicon

    args.stt_device = "mps"
    args.llm_device = "mps"
    args.qwen3_tts_device = "mps"
    # Additional handler-specific devices...

    return args

Activating this preset also switches the LLM backend to mlx-lm, which is optimized for Apple's MLX framework on M-series chips.

Global Device Propagation: overwrite_device_argument

When a global --device value is supplied, the overwrite_device_argument helper propagates it to every component field that ends with _device. This ensures consistent placement without manually setting each flag:

# src/speech_to_speech/s2s_pipeline.py (lines 317-331)

def overwrite_device_argument(args, device_value):
    for key in vars(args).keys():
        if key.endswith("_device"):
            setattr(args, key, device_value)
    return args

The global device is applied after component-specific flags, so --device will override any earlier settings unless used strategically.

CLI Options for Apple Silicon Device Placement

The following table summarizes how to control device placement on M1/M2/M3 Macs:

Option	Effect	Use Case
--mac-optimal-settings	Loads preset with MPS for all components + MLX-LM backend	Quick start with optimal defaults
--device {cpu,mps,cuda}	Sets global device for all components	Force uniform placement across handlers
--stt_device {cpu,mps}	Override STT device only	Offload transcription to CPU while keeping LLM/TTS on MPS
--llm_device {cpu,mps}	Override LLM device only	Reduce MPS memory pressure for large models
--qwen3_tts_device {cpu,mps}	Override TTS device only	Balance inference load across components

The argument definitions reside in src/speech_to_speech/arguments_classes/:

• Global device: ModuleArguments.device in module_arguments.py (lines 7-10)
• LLM device: LanguageModelArguments.llm_device in language_model_arguments.py (lines 8-10)
• TTS device: Qwen3TTSArguments.qwen3_tts_device in qwen3_tts_arguments.py (lines 13-16)

Practical Command-Line Examples

Example 1: Use the macOS Preset (Recommended)

The simplest approach for Apple Silicon:

python -m speech_to_speech --mac-optimal-settings

This applies MPS to all components and selects mlx-lm as the LLM handler.

Example 2: Explicitly Set Global Device to MPS

Equivalent to the preset, but without the MLX-LM switch:

python -m speech_to_speech --device mps

Under the hood, this triggers overwrite_device_argument to set stt_device, llm_device, qwen3_tts_device, and other component fields to "mps".

Example 3: Override Specific Component to CPU

Run TTS on CPU while keeping STT and LLM on MPS:

python -m speech_to_speech \
    --mac-optimal-settings \
    --qwen3_tts_device cpu

The preset loads first, then --qwen3_tts_device cpu overrides the TTS placement.

Example 4: Mixed Configuration Without Preset

Fine-grained control without loading any defaults:

python -m speech_to_speech \
    --stt_device mps \
    --llm_device cpu \
    --qwen3_tts_device mps

Example 5: Error Handling for Invalid Devices

The pipeline validates device choices. Attempting to use CUDA on macOS produces:

python -m speech_to_speech --device cuda

# Error: "Cannot use CUDA on macOS. Please set the device to 'cpu' or 'mps'."

This validation is implemented in s2s_pipeline.py (lines 305-306).

Configuration Precedence and Execution Flow

The argument parser resolves device placement in this order:

1. Parse raw CLI arguments
2. If --mac-optimal-settings, apply _mac_preset_defaults (sets all to MPS)
3. Apply any explicit component flags (--stt_device, --llm_device, etc.)
4. If --device provided, call overwrite_device_argument to replace all _device fields
5. Validate final device values (reject CUDA on macOS)

Understanding this flow helps prevent unexpected overrides. For example, placing --device before component flags in your command has no effect—the global device always wins if specified.

Performance Considerations on Apple Silicon

Component	MPS Benefit	When to Use CPU
STT (Whisper)	2-4x speedup vs CPU	Very short audio clips where overhead exceeds gain
LLM (MLX-LM)	Optimized for Apple Neural Engine	Model exceeds unified memory capacity
TTS (Qwen3)	Faster mel-spectrogram generation	Concurrent MPS workloads causing contention

Apple Silicon's unified memory architecture means MPS tensors share RAM with the CPU. Monitor memory pressure when running all three components on MPS simultaneously—offloading the largest model to CPU can improve stability.

Summary

• --mac-optimal-settings enables one-command MPS configuration with MLX-LM backend
• --device mps forces all components to MPS via overwrite_device_argument
• Component flags (--stt_device, --llm_device, --qwen3_tts_device) enable surgical overrides
• The configuration system in s2s_pipeline.py applies presets first, then explicit flags, then global device

Frequently Asked Questions

How do I check which device my components are actually running on?

Add the --verbose flag to your command. The pipeline logs device placement for each handler during initialization. You can also inspect the final parsed arguments by adding a debug print to src/speech_to_speech/s2s_pipeline.py before pipeline construction.

Can I use different devices for different TTS models?

Yes. The speech-to-speech repository uses model-specific argument classes. For Qwen3 TTS, use --qwen3_tts_device. If multiple TTS backends are supported in your version, each will have its own _{model}_tts_device flag defined in its respective arguments file.

What happens if MPS is unavailable on my Mac?

The pipeline falls back to CPU with a warning. However, explicit --device mps will raise an error if PyTorch was built without MPS support. Verify availability with python -c "import torch; print(torch.backends.mps.is_available())" before running the pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}