# How to Configure Multi-Language Support with Automatic Language Detection in Speech-to-Speech

> Configure multi language support for speech to speech using automatic language detection with lingua py and Parakeet TDT. Automatically detect 25 European languages.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-07-09

---

**Enable automatic language detection by installing `lingua-py` and using the Parakeet TDT STT handler with `language="auto"` or `language=None`, which activates the `_detect_language_from_text` method to identify 25 supported European languages from transcribed text.**

The huggingface/speech-to-speech repository provides real-time speech-to-speech translation capabilities with built-in support for configuring multi-language support with automatic language detection. While multiple STT handlers exist, only the **Parakeet TDT** implementation offers native automatic language detection (ALD) through the Lingua-py library. This guide explains how to enable and configure this feature using both the Python API and command-line interface.

## How Automatic Language Detection Works

The **Parakeet TDT handler** (`ParakeetTDTSTTHandler`) is currently the only STT handler in the repository that ships with built-in automatic language detection capabilities. Located in [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py), this handler integrates the **Lingua-py** library to infer ISO-639-1 language codes from transcribed text when users do not specify a target language.

### The Detection Mechanism

The handler initializes a lazy-loaded language detector via the `LINGUA_AVAILABLE` flag. Lines 72-89 pre-load language models at startup to eliminate latency on first use:

```python

# src/speech_to_speech/STT/parakeet_tdt_handler.py

if LINGUA_AVAILABLE:
    _lingua_iso_to_code = { … }
    _lingua_languages = [ … ]
    def _build_lingua_detector():
        return LanguageDetectorBuilder.from_languages(*_lingua_languages)\
                                      .with_preloaded_language_models()\
                                      .build()
    _lingua_detector = _build_lingua_detector()

```

### Detection Logic Implementation

The `_detect_language_from_text` method (lines 79-104) handles the actual detection logic. It skips very short utterances (fewer than 20 characters), runs the detector, maps the Lingua code back to the STS code table, and returns `None` if detection fails:

```python
def _detect_language_from_text(self, text: str) -> Optional[str]:
    if not LINGUA_AVAILABLE: …
    if not text or len(text.strip()) < 20: …
    detected = _lingua_detector.detect_language_of(text)
    if detected is None: return None
    code = detected.iso_code_639_1.name.lower()
    return {v: k for k, v in _LINGUA_CODE_MAP.items()}.get(code, code)

```

## Installation Requirements

### Installing the Lingua-py Dependency

Automatic language detection requires the optional **Lingua-py** package. Without this dependency, the handler cannot perform language inference and will skip detection entirely.

```bash
pip install lingua-py

```

## Configuration Methods

You can activate multi-language support through either the Python API or the command-line interface. Both methods require setting the `language` parameter to `"auto"` or `None` to trigger the detection pipeline.

### Python API Configuration

In [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py), the `SpeechToSpeechPipeline` class accepts a `language` argument that it passes to the handler. Set `language=None` to enable automatic detection:

```python
from speech_to_speech.s2s_pipeline import SpeechToSpeechPipeline

# language=None defaults to auto-detection

pipeline = SpeechToSpeechPipeline(
    stt_handler="parakeet",
    language=None,            # activates automatic language detection

    # other arguments …

)

```

The pipeline internally creates a `ParakeetTDTSTTHandler` instance and passes the `language` argument defined in [`src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py).

### Command Line Configuration

Use the [`scripts/listen_and_play_realtime.py`](https://github.com/huggingface/speech-to-speech/blob/main/scripts/listen_and_play_realtime.py) entry point with the `--language auto` flag:

```bash
python -m speech_to_speech.scripts.listen_and_play_realtime \
    --stt-handler parakeet \
    --language auto          # auto-detect per utterance

```

## Language Selection Priority

After transcription in `_process_mlx` and `_process_nano_parakeet`, the handler determines the final language code using this priority order:

1. **User-provided language**: If `self.start_language` is set and not equal to `"auto"`, the handler uses this value exclusively.
2. **Detected language**: If auto-detection is active, the handler calls `_detect_language_from_text` on the transcribed text.
3. **Fallback language**: If detection fails or returns `None`, the system uses `self.last_language` (the most recently successful detection).

```python
if self.start_language and self.start_language != "auto":
    language_code = self.start_language
else:
    detected_lang = self._detect_language_from_text(pred_text)
    language_code = detected_lang or self.last_language

```

## Supported Languages and Constraints

The **Parakeet TDT handler** supports **25 European languages** defined in the `SUPPORTED_LANGUAGES` constant. The Lingua-py detector is configured specifically for this subset, ensuring returned ISO-639-1 codes (e.g., `en`, `fr`, `de`, `es`) are always compatible with the handler's capability set. The system prints detected language codes to the console output (lines 53-55 of the handler file), allowing you to verify detection accuracy in real-time.

## Summary

- Install `lingua-py` to enable the automatic language detection feature
- Use the `parakeet` STT handler located in [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py)
- Set `language="auto"` or `language=None` to activate detection per utterance
- Detection requires a minimum of 20 characters of transcribed text to trigger
- The system maintains state in `self.last_language` to handle ambiguous or short utterances

## Frequently Asked Questions

### Which STT handlers support automatic language detection?

Only the **Parakeet TDT handler** (`parakeet`) supports automatic language detection. Other handlers in the repository require explicit language configuration. The detection logic resides in the `_detect_language_from_text` method of [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py).

### What happens if the detected language is not supported?

If Lingua-py cannot determine the language with confidence, or if the transcribed text contains fewer than 20 characters, the method returns `None`. The handler then falls back to `self.last_language`, which stores the most recently successful detection to maintain continuity across short responses.

### Can I lock the pipeline to a specific language?

Yes. Pass an explicit ISO-639-1 code (e.g., `language="de"` or `--language de`) to bypass automatic detection entirely. When a specific language is provided, the handler skips the `_detect_language_from_text` call and forces that language for all subsequent transcriptions.

### Where is the language detection configured in the source code?

The core detection implementation is in [`src/speech_to_speech/STT/parakeet_tdt_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/STT/parakeet_tdt_handler.py) (lines 72-104). CLI arguments are defined in [`src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/parakeet_tdt_arguments.py), and the pipeline wiring occurs in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py). The [`README.md`](https://github.com/huggingface/speech-to-speech/blob/main/README.md) (line 468) also documents this feature under "Automatic language detection."