# Supported Languages for olmocr: HTML-Based Language Detection and Normalization > Discover supported languages for OLMOCR. Learn how it detects and normalizes languages using HTML lang attributes, defaulting to English when needed. - Repository: [Ai2/olmocr](https://github.com/allenai/olmocr) - Tags: api-reference - Published: 2026-07-07 --- **OLMOCR determines the supported language of a document by extracting the ISO-639-1 code from the HTML `` attribute, normalizing it to a two-letter code (e.g., `pt-BR` → `pt`), and storing it in the `primary_language` metadata field, defaulting to English (`en`) when the attribute is absent.** The Allen AI **olmocr** pipeline processes multilingual documents by detecting language codes embedded in HTML templates. Understanding the supported languages for olmocr is straightforward: the system extracts the `lang` attribute from the `` tag, normalizes regional variants to standard two-letter ISO-639-1 codes, and records this value in the document metadata. This design ensures compatibility with any valid ISO-639-1 language code while maintaining consistency across the processing pipeline. ## How olmocr Detects Document Languages OLMOCR relies on the **HTML `lang` attribute** to establish document language. During the synthesis phase, the `extract_html_metadata` function in [`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py) parses the HTML structure to locate the `` tag and extract its language property. The detection process follows these steps: 1. **Parse the HTML** to locate the root `` element 2. **Extract the `lang` attribute** value (e.g., `pt-BR`, `en`, `es`) 3. **Normalize to ISO-639-1** by stripping regional variants to two-letter codes (e.g., `pt-BR` becomes `pt`) 4. **Store in metadata** within the `primary_language` field 5. **Default to English** (`en`) if the attribute is missing or unparsable This mechanism allows olmocr to support any valid ISO-639-1 language code, including `ja` (Japanese), `zh` (Chinese), and `ru` (Russian), provided the HTML source correctly specifies the language. ## Verified Language Support and Test Coverage The test suite in [`tests/test_mine_html_templates_metadata.py`](https://github.com/allenai/olmocr/blob/main/tests/test_mine_html_templates_metadata.py) explicitly validates language extraction for several major languages. The following codes are verified in the current codebase: - **English** (`en`) – Validated in `test_extract_metadata_table_heavy_document` - **Portuguese** (`pt`) – Validated in `test_extract_metadata_portuguese_document` - **Spanish** (`es`) – Validated in `test_extract_metadata_image_heavy_document` - **German** (`de`) – Validated in `test_extract_metadata_empty_body` - **French** (`fr`) – Referenced in pipeline tests within [`tests/test_grpo.py`](https://github.com/allenai/olmocr/blob/main/tests/test_grpo.py) While these languages have explicit test coverage, the normalization logic accepts any standard two-letter ISO-639-1 code. ## Code Implementation and Examples The language detection logic resides in [`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py) and is exercised by unit tests in [`tests/test_mine_html_templates_metadata.py`](https://github.com/allenai/olmocr/blob/main/tests/test_mine_html_templates_metadata.py). ### Detecting Portuguese Documents ```python from olmocr.synth.mine_html_templates import extract_html_metadata html_pt = '

Texto em português.

' metadata_pt = extract_html_metadata(html_pt) print(metadata_pt["primary_language"]) # Output: "pt" ``` ### Normalizing Regional Variants ```python from olmocr.synth.mine_html_templates import extract_html_metadata # Brazilian Portuguese normalizes to standard Portuguese html_pt_br = '

Texto em português brasileiro.

' metadata_pt_br = extract_html_metadata(html_pt_br) print(metadata_pt_br["primary_language"]) # Output: "pt" ``` ### Handling Missing Language Attributes ```python from olmocr.synth.mine_html_templates import extract_html_metadata # Missing lang attribute defaults to English html_no_lang = '

No language attribute.

' metadata_default = extract_html_metadata(html_no_lang) print(metadata_default["primary_language"]) # Output: "en" ``` ## Key Implementation Files Language detection is implemented across the following files: - **[`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py)** – Contains the `extract_html_metadata` function that parses HTML and normalizes language codes to ISO-639-1. - **[`tests/test_mine_html_templates_metadata.py`](https://github.com/allenai/olmocr/blob/main/tests/test_mine_html_templates_metadata.py)** – Unit tests verifying extraction for Portuguese, Spanish, German, and English documents. - **[`tests/test_grpo.py`](https://github.com/allenai/olmocr/blob/main/tests/test_grpo.py)** – Integration tests confirming the pipeline propagates `primary_language` values including French and German. ## Summary - OLMOCR detects document languages from the HTML `` attribute using the `extract_html_metadata` function. - Regional language codes (e.g., `pt-BR`, `en-US`) normalize to two-letter ISO-639-1 codes (e.g., `pt`, `en`). - The system supports any valid ISO-639-1 language code, not just the explicitly tested languages. - Missing or unparsable language attributes default to English (`en`). - Language metadata is stored in the `primary_language` field of the document metadata object. ## Frequently Asked Questions ### What happens if the HTML lang attribute is missing? If the `` tag lacks a `lang` attribute or the value cannot be parsed, olmocr defaults to English (`en`) as the `primary_language` according to the implementation in [`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py). ### Does olmocr support regional language variants like pt-BR or en-US? Yes. The `extract_html_metadata` function normalizes regional variants by extracting only the first two letters of the language code. For example, `pt-BR` becomes `pt` and `en-US` becomes `en`, maintaining compatibility with the ISO-639-1 standard. ### How can I verify language detection is working correctly? Run the unit tests in [`tests/test_mine_html_templates_metadata.py`](https://github.com/allenai/olmocr/blob/main/tests/test_mine_html_templates_metadata.py), which validate extraction for Portuguese, Spanish, German, and English. Additional integration tests in [`tests/test_grpo.py`](https://github.com/allenai/olmocr/blob/main/tests/test_grpo.py) verify that the `primary_language` field propagates correctly through the complete pipeline. ### Is language detection automatic or does it require configuration? Language detection is automatic. OLMOCR reads the `lang` attribute from the HTML `` tag during document processing without requiring additional configuration or manual language selection.