# How to Convert Speech from One Language to Another Using the Hugging Face Speech-to-Speech Library

> Learn how to convert speech from one language to another using the Hugging Face speech-to-speech library. Configure the S2SPipeline for seamless language translation and synthesis.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-01

---

**You can convert speech from one language to another by configuring the `S2SPipeline` with a Whisper STT handler for source detection, a language model handler for translation, and a Qwen-3 TTS handler for synthesis, passing the target `language_code` through `runtime_config` to propagate language metadata across all pipeline stages.**

The Hugging Face `speech-to-speech` repository provides an open-source framework for real-time speech translation. By orchestrating three specialized handlers through the `S2SPipeline` class defined in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py), you can convert spoken audio from a detected source language to synthesized speech in a specified target language with minimal latency.

## The Three-Stage Translation Architecture

The library implements a full-duplex pipeline that processes audio through three sequential stages, each communicating via asynchronous queues and typed message objects from [`src/speech_to_speech/pipeline/messages.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/messages.py).

**Speech-to-Text (STT)**: The `WhisperSTTHandler` in [`src/speech_to_speech/STT/whisper_stt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/whisper_stt_handler.py) transcribes incoming audio into a `Transcription` message. When configured with `language="auto"`, Whisper detects the source language and stores the ISO code (e.g., `"en"`, `"fr"`) in `handler.last_language`, which attaches to the message's `language_code` field.

**Language Model Translation**: The `LanguageModelHandler` (utilizing [`src/speech_to_speech/LLM/language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/language_model.py)) receives the transcription and generates a translation. It emits `GenerateResponseRequest` and `LLMResponseChunk` messages carrying the target `language_code` specified in your configuration, effectively translating content while updating language metadata for downstream handlers.

**Text-to-Speech (Synthesis)**: The `Qwen3TTSHandler` in [`src/speech_to_speech/TTS/qwen3_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/qwen3_tts_handler.py) consumes the translated text along with the target `language_code`, synthesizing audio that matches the desired output language and packaging it into an `AudioOutput` message.

## Configuring Source and Target Languages

Language flow is controlled through the `runtime_config` dictionary passed to `S2SPipeline`. The pipeline distinguishes between automatic source detection and explicit target specification.

**Source Language Detection**

When instantiating `WhisperSTTArguments`, set `language="auto"` to enable automatic language detection. The handler stores the detected code in `handler.last_language`, which propagates to the `Transcription` message's `language_code` attribute. For explicit control, pass a specific ISO code such as `language="de"` to skip detection and force German recognition.

**Target Language Specification**

Set the `language_code` key in your `runtime_config` to the desired target ISO code (e.g., `"es"` for Spanish). This value flows to the `LanguageModelHandler`, instructing it to generate `LLMResponseChunk` objects in that language, and subsequently to the TTS handler for voice synthesis.

## Implementation Examples

### High-Level Pipeline Configuration

The following example demonstrates configuring the pipeline for English-to-Spanish translation using the argument classes and `S2SPipeline`:

```python
from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.arguments_classes.whisper_stt_arguments import WhisperSTTArguments
from speech_to_speech.arguments_classes.qwen3_tts_arguments import Qwen3TTSArguments
from speech_to_speech.arguments_classes.language_model_arguments import LanguageModelArguments

# Configure runtime with target language

runtime_config = {
    "language_code": "es",  # Target: Spanish

    "stt": WhisperSTTArguments(language="auto"),  # Auto-detect source

    "tts": Qwen3TTSArguments(),
    "lm": LanguageModelArguments(),
}

pipeline = S2SPipeline(runtime_config)
pipeline.run()  # Starts listening and speaking loop

```

### Programmatic Audio Processing

For batch processing or custom audio sources, feed raw bytes directly without using the microphone loop:

```python

# Initialize pipeline (English input → French output)

config = {
    "language_code": "fr",
    "stt": WhisperSTTArguments(language="en"),
    "tts": Qwen3TTSArguments(),
    "lm": LanguageModelArguments(),
}
pipeline = S2SPipeline(config)

# Process audio file

with open("input.wav", "rb") as f:
    audio_chunk = f.read()
    
pipeline.feed_audio(audio_chunk)
output_audio = pipeline.get_output_audio()

```

### Fixed Source Language Configuration

To bypass automatic detection and reduce latency when the source language is known:

```python

# Explicit German to English translation

runtime_config = {
    "language_code": "en",
    "stt": WhisperSTTArguments(language="de"),  # Fixed source

    "tts": Qwen3TTSArguments(),
    "lm": LanguageModelArguments(),
}
pipeline = S2SPipeline(runtime_config)
pipeline.run()

```

## Key Source Files and Components

Understanding these core files helps debug and extend the translation pipeline:

- [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py): Contains the `S2SPipeline` class that initializes handler queues and manages the event loop.
- [`src/speech_to_speech/pipeline/messages.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/messages.py): Defines message types (`Transcription`, `TTSInput`, `AudioOutput`) that carry `language_code` metadata between stages.
- [`src/speech_to_speech/pipeline/events.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/events.py): Implements typed events like `TranscriptionCompletedEvent` that trigger stage transitions.
- [`src/speech_to_speech/STT/whisper_stt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/whisper_stt_handler.py): Handles speech recognition and source language detection.
- [`src/speech_to_speech/LLM/language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/language_model.py): Provides translation capabilities via the `LanguageModelHandler`.
- [`src/speech_to_speech/TTS/qwen3_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/qwen3_tts_handler.py): Synthesizes speech for the target language.
- [`scripts/listen_and_play.py`](https://github.com/huggingface/speech-to-speech/blob/main/scripts/listen_and_play.py): Reference implementation demonstrating microphone input to speaker output.

## Summary

- The `S2SPipeline` orchestrates three stages (STT, LM, TTS) to convert speech between languages.
- Source language is detected automatically by Whisper or set explicitly via `WhisperSTTArguments`.
- Target language is specified through `runtime_config["language_code"]` and propagated through message objects.
- Handlers normalize language aliases (e.g., `"English"` to `"en"`) and fallback to `last_language` if unsupported codes are encountered.
- The pipeline supports both real-time streaming and programmatic batch processing.

## Frequently Asked Questions

### How does the pipeline detect the source language?

The `WhisperSTTHandler` uses OpenAI's Whisper model with `language="auto"` to detect the spoken language from audio features. It stores the ISO 639-1 code in the `language_code` field of the `Transcription` message, which subsequent handlers receive for processing.

### Can I use different models for STT or TTS?

Yes. While the examples use `WhisperSTTHandler` and `Qwen3TTSHandler`, the pipeline architecture accepts any handler implementing the base interfaces. Replace `WhisperSTTArguments` with your custom STT arguments class or substitute the TTS handler by modifying the `tts` key in `runtime_config` to point to alternative implementations.

### What happens if I specify an unsupported target language?

The handlers normalize language aliases and validate against supported codes. If an unsupported code is encountered, the system falls back to the `last_language` attribute stored in the handler state, ensuring the pipeline continues processing rather than failing.

### Is this processing suitable for real-time conversations?

Yes. The `S2SPipeline` uses asynchronous queues and streaming handlers to minimize latency between speech input and audio output. The [`scripts/listen_and_play.py`](https://github.com/huggingface/speech-to-speech/blob/main/scripts/listen_and_play.py) example demonstrates this by processing microphone input and playing synthesized responses with minimal delay, suitable for interactive dialogue.