# How to Set Up Multi-Language Support with Automatic Language Detection in Speech-to-Speech

> Enable automatic language detection for speech-to-speech translation across 25 European languages. Install lingua-py, use Parakeet TDT handler, and set language to auto.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-07-08

---

**Install `lingua-py`, use the Parakeet TDT handler, and set `language="auto"` (or omit the language parameter) to enable automatic language detection across 25 supported European languages.**

The Hugging Face `speech-to-speech` repository provides real-time speech-to-speech translation pipelines with built-in multi-language support. When you need the system to automatically identify the spoken language without manual configuration, the **Parakeet TDT STT handler** integrates the Lingua-py library for automatic language detection (ALD) across 25 European languages.

## Prerequisites: Installing the Language Detection Dependency

Automatic language detection requires the optional `lingua-py` package. The handler implements a lazy-loading mechanism via the `LINGUA_AVAILABLE` flag, but you must install the dependency before starting the pipeline.

```bash
pip install lingua-py

```

Without this package, the handler will skip detection and fall back to the default language or the last detected language.

## How Automatic Language Detection Works

The automatic language detection system operates entirely within the `ParakeetTDTSTTHandler` class in [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py). The handler detects language in real-time after each transcription using the following logic:

- **Detection trigger**: When `language=None` or `language="auto"` is passed to the handler, the system calls `_detect_language_from_text` after generating the transcription text.
- **Language library**: The handler uses **Lingua-py**, a high-accuracy natural language detection library, to infer ISO-639-1 language codes from the transcribed text.
- **Supported languages**: The system supports 25 European languages defined in the `SUPPORTED_LANGUAGES` list, ensuring the detector returns only valid, supported codes.
- **Pre-loading**: To avoid latency on first use, the handler pre-loads language models at startup using `LanguageDetectorBuilder.from_languages(*_lingua_languages).with_preloaded_language_models()`.

## Configuration Methods

You can enable multi-language support with automatic detection via the Python API or the command-line interface.

### Python API Configuration

When constructing the pipeline in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py), pass `language=None` or omit the parameter to trigger automatic detection:

```python
from speech_to_speech.s2s_pipeline import SpeechToSpeechPipeline

# language=None enables automatic detection

pipeline = SpeechToSpeechPipeline(
    stt_handler="parakeet",
    language=None,  # or "auto"

    # other arguments...

)

```

The `SpeechToSpeechPipeline` internally instantiates the `ParakeetTDTSTTHandler` and passes the language argument defined in [`src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py).

### Command-Line Interface

Use the [`listen_and_play_realtime.py`](https://github.com/huggingface/speech-to-speech/blob/main/listen_and_play_realtime.py) script with the `--language auto` flag:

```bash
python -m speech_to_speech.scripts.listen_and_play_realtime \
    --stt-handler parakeet \
    --language auto

```

If you want to lock to a specific language and disable automatic detection, provide an ISO-639-1 code (e.g., `--language de` or `--language fr`).

## Implementation Details: The Detection Pipeline

The detection logic follows a specific priority order to determine the final language code for each utterance.

### Language Detection Method

The `_detect_language_from_text` method (lines 79-104 in [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py)) implements the core detection logic:

```python
def _detect_language_from_text(self, text: str) -> Optional[str]:
    if not LINGUA_AVAILABLE:
        return None
    if not text or len(text.strip()) < 20:
        return None  # Skip very short utterances

    
    detected = _lingua_detector.detect_language_of(text)
    if detected is None:
        return None
    
    code = detected.iso_code_639_1.name.lower()
    # Map Lingua code back to STS code table

    return {v: k for k, v in _LINGUA_CODE_MAP.items()}.get(code, code)

```

The handler skips detection for utterances shorter than 20 characters to ensure sufficient text for accurate classification.

### Language Resolution Priority

After transcription in `_process_mlx` or `_process_nano_parakeet`, the handler resolves the language code in this order:

1. **User-provided language**: If `self.start_language` is set and not equal to `"auto"`, the handler uses this value and skips detection.
2. **Auto-detected language**: If the user requested auto-detection, the handler calls `_detect_language_from_text`.
3. **Last known language**: If detection fails (returns `None`), the system falls back to `self.last_language`, persisting the last successful detection across utterances.

```python
if self.start_language and self.start_language != "auto":
    language_code = self.start_language
else:
    detected_lang = self._detect_language_from_text(pred_text)
    language_code = detected_lang or self.last_language

```

## Handling Language Fallbacks and Edge Cases

The handler maintains state across utterances through the `self.last_language` attribute. This ensures that if the detector fails to identify a short or ambiguous phrase, the system continues using the previously detected language rather than defaulting to a potentially incorrect code.

To force a specific language and disable automatic detection, specify the language code explicitly:

```python

# Force German language, skip auto-detection

pipeline = SpeechToSpeechPipeline(
    stt_handler="parakeet",
    language="de"
)

```

## Summary

- **Install `lingua-py`** to enable automatic language detection capabilities.
- **Use the Parakeet TDT handler** (`--stt-handler parakeet`) as it is the only handler shipping with built-in automatic language detection.
- **Set `language="auto"` or `language=None`** to trigger detection, or omit the parameter entirely.
- **Detection requires 20+ characters** of transcribed text; shorter utterances fall back to the last detected language.
- **Configure via Python** using `SpeechToSpeechPipeline` or via **CLI** using [`listen_and_play_realtime.py`](https://github.com/huggingface/speech-to-speech/blob/main/listen_and_play_realtime.py).
- **Supported languages** include 25 European languages mapped to ISO-639-1 codes in the handler's `SUPPORTED_LANGUAGES` list.

## Frequently Asked Questions

### What is the minimum text length required for automatic language detection?

The `_detect_language_from_text` method in [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py) skips detection for any utterance shorter than 20 characters. This threshold ensures Lingua-py has sufficient content to accurately classify the language. If the text is too short, the handler returns `None` and falls back to `self.last_language`.

### Can I use automatic language detection with other STT handlers?

No. According to the source code, only the **Parakeet TDT handler** (`ParakeetTDTSTTHandler`) implements automatic language detection. Other handlers in the repository do not include the `_detect_language_from_text` method or the Lingua-py integration. You must specify `stt_handler="parakeet"` to use this feature.

### How does the system handle language detection failures?

If Lingua-py returns `None` or the text is too short, the handler falls back to `self.last_language`, which stores the most recently successful detection. This state persists across utterances, ensuring continuity in multi-turn conversations even when individual phrases are too brief for reliable detection.

### What languages are supported for automatic detection?

The handler supports 25 European languages defined in the `SUPPORTED_LANGUAGES` list within [`parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/parakeet_tdt_handler.py). The detection system uses ISO-639-1 codes (e.g., `en`, `fr`, `de`, `es`) and maps them through the Lingua-py library's `iso_code_639_1` enumeration.