# How the Real-Time Voice Cloning Synthesizer Handles Out-of-Vocabulary Words and Unknown Characters

> Discover how the Real-Time Voice Cloning synthesizer tackles OOV words and unknown characters using text preprocessing, transliteration, and ARPAbet for robust speech synthesis.

- Repository: [Corentin Jemine/Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)
- Tags: internals
- Published: 2026-03-05

---

**The Real-Time Voice Cloning synthesizer handles out-of-vocabulary (OOV) words and unknown characters by silently filtering unsupported symbols during text preprocessing, while providing transliteration cleaners and ARPAbet phonetic notation as fallback mechanisms to ensure robust speech synthesis.**

The CorentinJ/Real-Time-Voice-Cloning repository implements a Tacotron-based spectrogram generator that operates on character-level inputs. Because the model is trained on a fixed vocabulary of symbols, any character outside this predefined set is considered out-of-vocabulary. Understanding how the `synthesizer` module processes these edge cases is critical for generating intelligible speech from arbitrary text inputs.

## The Symbol Vocabulary Definition

The synthesizer's vocabulary is hardcoded in [`synthesizer/utils/symbols.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/symbols.py) as a global `symbols` list containing uppercase and lowercase ASCII letters, punctuation marks, the space character, and special tokens. The padding token is represented by an underscore `"_"` and the end-of-sentence (EOS) token by a tilde `"~"`.

During inference, the `text_to_sequence` function in [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py) converts input strings into numeric IDs by mapping each character to its index in this `symbols` list. This strict mapping ensures the model receives only integers it was trained to interpret, but it also creates a boundary where unsupported characters must be handled explicitly.

## Text Cleaning and Transliteration

Before symbol conversion, raw input passes through a cleaner pipeline defined in [`synthesizer/utils/cleaners.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/cleaners.py). The `transliteration_cleaners` and `english_cleaners` options invoke `convert_to_ascii`, which uses the `unidecode` library to map accented or non-ASCII characters to their closest ASCII equivalents.

```python
from synthesizer.inference import Synthesizer
from pathlib import Path

synth = Synthesizer(Path("saved_models/default/synthesizer.pt"))

# Accented characters are transliterated before symbol mapping

text = "Café à la mode."
wav = synth.synthesize(text, ["transliteration_cleaners"])

# é → e, à → a via unidecode in cleaners.py lines 62-64

```

This preprocessing step reduces OOV occurrences by normalizing Unicode characters into the supported ASCII subset, preventing silent deletion of semantic content.

## Silent Filtering of Unknown Characters

When characters survive cleaning but remain outside the `symbols` list, the `_symbols_to_sequence` function in [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py) applies a strict filter. The helper `_should_keep_symbol` checks membership in the vocabulary and returns `False` for any unknown character, causing it to be skipped entirely during sequence generation.

```python

# synthesizer/utils/text.py lines 74-76

def _should_keep_symbol(s):
    return s in _symbol_to_id and s is not '_' and s is not '~'

```

Consequently, emojis, mathematical symbols, or foreign scripts that bypass the cleaners are removed without raising errors. The resulting sequence behaves as if the characters were never present, which can alter pronunciation if the removed characters carried phonetic weight.

## ARPAbet Phonetic Fallback

For words with non-standard pronunciation or unsupported spellings, the synthesizer accepts ARPAbet notation wrapped in curly braces `{...}`. In [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py) (lines 29-38), the text parser detects these braces and extracts phonemes, prefixing each with an "@" symbol (e.g., `@AH0`, `@HH`). These ARPAbet symbols are dynamically added to the valid symbol set at inference time, allowing phonetic spell-outs to bypass the standard character vocabulary entirely.

```python

# Explicit phonetic input guarantees pronunciation regardless of spelling

text = "The city {HH AW1 S T AH0 N} is large."
wav = synth.synthesize(text, ["speaker_0"])

# ARPAbet symbols are preserved and converted to IDs despite being non-standard characters

```

This mechanism ensures that proper nouns or technical terms with unpredictable orthography can still be synthesized correctly by providing their phonetic transcription directly.

## Practical Implementation Examples

The following patterns demonstrate the complete OOV handling pipeline in practice:

```python
from synthesizer.inference import Synthesizer
from pathlib import Path

synth = Synthesizer(Path("saved_models/default/synthesizer.pt"))

# Example 1: Standard English (all characters supported)

wav = synth.synthesize("Hello, world!", ["speaker_0"])

# Example 2: Emoji silently filtered by _should_keep_symbol

# Input "I love pizza 🍕!" becomes equivalent to "I love pizza !"

wav_oov = synth.synthesize("I love pizza 🍕!", ["speaker_0"])

# Example 3: Non-ASCII handled via transliteration_cleaners

# Uses cleaners.py convert_to_ascii before symbol mapping

wav_clean = synth.synthesize("naïve résumé", ["transliteration_cleaners"])

# Example 4: ARPAbet override for guaranteed pronunciation

wav_phonetic = synth.synthesize("Say {S EH1 D}", ["speaker_0"])

```

## Summary

- **Strict vocabulary**: The synthesizer uses a fixed `symbols` list in [`synthesizer/utils/symbols.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/symbols.py) containing only ASCII letters, punctuation, and special tokens.
- **Preprocessing**: `transliteration_cleaners` in [`synthesizer/utils/cleaners.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/cleaners.py) converts accented characters to ASCII using `unidecode`, reducing OOV instances.
- **Silent removal**: The `_should_keep_symbol` helper in [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py) filters any remaining unknown characters without throwing errors.
- **Phonetic bypass**: ARPAbet notation wrapped in `{}` allows direct phoneme input, bypassing character-level OOV issues entirely.

## Frequently Asked Questions

### What happens if I include emojis or special Unicode characters in the input text?

Emojis and unsupported Unicode characters are silently removed during the `_symbols_to_sequence` conversion in [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py). The `_should_keep_symbol` function returns `False` for any character not present in the predefined `symbols` list, so these characters never reach the Tacotron model and do not generate errors, though they may leave gaps in the prosody where they were removed.

### Can the synthesizer handle accented characters or non-English alphabets?

Accented characters are handled only if you use the `transliteration_cleaners` or similar cleaners that invoke `convert_to_ascii` in [`synthesizer/utils/cleaners.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/cleaners.py). This function uses the `unidecode` library to strip accents (e.g., "é" becomes "e") before symbol mapping. True non-English alphabets (Cyrillic, Arabic, etc.) are treated as OOV and filtered out unless you implement custom cleaners or use ARPAbet notation for phonetic representation.

### How do I force a specific pronunciation for a word the synthesizer mispronounces?

Wrap the word in ARPAbet phoneme notation using curly braces. For example, input `{HH AW1 S T AH0 N}` instead of "Houston". The parser in [`synthesizer/utils/text.py`](https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/main/synthesizer/utils/text.py) extracts these phonemes, prefixes them with "@", and includes them in the valid symbol set, ensuring the model processes the explicit phonetic instructions rather than attempting character-level spelling.

### Will unknown characters cause the synthesis to fail or raise exceptions?

No, unknown characters do not raise exceptions. The `_symbols_to_sequence` function safely skips any character failing the `_should_keep_symbol` check, resulting in a shorter input sequence. The synthesis proceeds with the remaining valid characters, though the resulting audio may sound incorrect if the removed characters were essential to the intended pronunciation.