# How to Configure Multi-Language Support with Automatic Language Detection in Speech-to-Speech

> Configure multi-language speech-to-speech translation with automatic language detection for 25+ European languages. Install lingua-py and set language to None for STT handlers.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-10

---

**Install the optional `lingua-py` package and initialize any supported STT handler (such as `ParakeetTDTSTTHandler`) with `language=None` to enable real-time automatic language detection across 25+ European languages.**

The Hugging Face `speech-to-speech` repository provides a modular pipeline for real-time speech-to-text-to-speech conversion. Configuring multi-language support with automatic language detection allows the system to dynamically identify spoken languages from audio input and route transcriptions to appropriate TTS voices and LLM prompts without manual configuration.

## Architecture Overview

The library implements multi-language support through **STT handlers** that optionally detect the input language using the `lingua` library. When enabled, the detection pipeline follows this flow:

- The **STT Handler** (e.g., `ParakeetTDTSTTHandler`) transcribes audio and runs `_detect_language_from_text` on the result.
- The **Language Detector** (`lingua.LanguageDetector`) infers the ISO-639-1 language code from the transcription text.
- The **Pipeline** (`S2SPipeline`) receives a `Transcription` object containing the `language_code` and propagates it to downstream components.

The core implementation resides in [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py). This handler defines `SUPPORTED_LANGUAGES` covering 25 European languages and implements `_build_lingua_detector` to initialize the detector at startup. The private method `_detect_language_from_text` handles edge cases such as mapping Lingua's `"nb"` (Norwegian Bokmål) to the library's internal `"no"` code.

Other compatible handlers include [`whisper_stt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/whisper_stt_handler.py), [`mlx_audio_whisper_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/mlx_audio_whisper_handler.py), and [`lightning_whisper_mlx_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/lightning_whisper_mlx_handler.py), all of which expose a `language_code` field in their transcription results.

## Installing the Language Detector

Automatic detection requires the optional `lingua-py` dependency. Without this package, handlers default to the user-specified language or `"en"`.

```bash
pip install lingua-py

```

## Enabling Automatic Detection

To activate multi-language support, configure your STT handler without specifying a static language code.

### Programmatic Configuration

Instantiate `S2SPipeline` with an STT handler and set `language=None` during setup:

```python
from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.STT.parakeet_tdt_handler import ParakeetTDTSTTHandler

# Initialize pipeline

pipeline = S2SPipeline(
    stt_handler=ParakeetTDTSTTHandler(),
    tts_handler=None,
    llm_handler=None,
)

# Configure for automatic detection

pipeline.stt_handler.setup(
    device="auto",
    compute_type="float16",
    language=None,  # Triggers automatic detection

)

# Process audio

transcription = pipeline.transcribe(audio_path="sample.wav")
print(f"Detected: {transcription.language_code}")
print(f"Text: {transcription.text}")

```

When `language=None`, the handler invokes `_detect_language_from_text` on each transcription, loading the Lingua detector at startup via `_build_lingua_detector`.

### CLI Configuration

Run the pipeline from the command line without the `--language` flag:

```bash
speech-to-speech \
    --stt parakeet-tdt \
    --device auto \
    --audio-file samples/multi_lang.wav

```

The CLI forwards arguments to `ParakeetTDTSTTHandler.setup`, and the omission of `--language` enables the detection routine.

## Customizing Language Support

The default `SUPPORTED_LANGUAGES` list in [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py) covers common European languages. To add support for additional languages, modify the constant in the source file:

```python

# In src/speech_to_speech/STT/parakeet_tdt_handler.py

SUPPORTED_LANGUAGES = [
    "en", "de", "fr", "es", "it", "pt", "nl", "pl", "ru",
    "zh",  # Added Chinese support

    # ... additional ISO-639-1 codes

]

```

After editing, restart the pipeline. The detector will consider the new language during inference.

## Integration with Pipeline Components

The detected `language_code` propagates through the entire pipeline:

- **LLM Prompting**: The `VoicePrompt` and `TextPrompt` classes in [`speech_to_speech/LLM/voice_prompt.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/LLM/voice_prompt.py) and [`speech_to_speech/LLM/text_prompt.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/LLM/text_prompt.py) adapt prompts based on the detected language.
- **TTS Selection**: TTS handlers such as [`kokoro_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/kokoro_handler.py) use language-specific mappings (e.g., `kokoro_lang_code`) to select appropriate voices for the detected language.
- **Logging**: Handlers log detection events via `logger.info` and `logger.warning`, enabling debugging of detection failures.

## Summary

- Install `lingua-py` to enable the automatic language detection capability.
- Set `language=None` in `ParakeetTDTSTTHandler.setup()` or omit `--language` from CLI commands to trigger detection.
- The system supports 25 European languages by default via the `SUPPORTED_LANGUAGES` constant in [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py).
- Detected language codes flow through `S2SPipeline` to influence TTS voice selection and LLM prompting automatically.
- Extend support to non-European languages by editing the `SUPPORTED_LANGUAGES` list in the handler source code.

## Frequently Asked Questions

### Which STT handlers support automatic language detection?

The `ParakeetTDTSTTHandler` in [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py) provides built-in detection via the `_detect_language_from_text` method. The [`whisper_stt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/whisper_stt_handler.py), [`mlx_audio_whisper_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/mlx_audio_whisper_handler.py), and [`lightning_whisper_mlx_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/lightning_whisper_mlx_handler.py) handlers also support automatic detection when `lingua-py` is installed and `language` is set to `None`.

### What happens if I don't install `lingua-py`?

Without the `lingua-py` package installed, the STT handlers skip the detection routine entirely. The system defaults to the language specified by the user during setup, or falls back to `"en"` (English) if no language is provided.

### Can I restrict automatic detection to specific languages only?

Yes. Edit the `SUPPORTED_LANGUAGES` list in your chosen handler file (e.g., [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py)) to include only the ISO-639-1 codes you want to detect. The `lingua.LanguageDetector` will only consider these languages during inference, improving accuracy and reducing latency.

### How does the detected language affect the TTS output?

The `language_code` field in the `Transcription` object is passed through `S2SPipeline` to the TTS handler. For example, [`kokoro_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/kokoro_handler.py) maps this code to a specific `kokoro_lang_code` and voice, ensuring the response is synthesized in the same language as the input audio.