How the readme-translations.py Script Manages the i18n Landing Page Pipeline

The readme-translations.py script stores hand-crafted translation tables that map exact English text blocks to localized versions, while build_readme_i18n.py scans the canonical README.md, identifies translatable spans using regex patterns, and generates language-specific landing pages in i18n/<lang>/README.md.

Managing multilingual documentation for open-source repositories requires a pipeline that preserves Markdown formatting while enabling community-driven translations. The rohitg00/ai-engineering-from-scratch repository implements a loss-less internationalization system using two coordinated Python utilities that separate translation data from rendering logic. The readme-translations.py file serves as the authoritative source for all localized content, enabling incremental language additions without breaking the build process.

The Two-Script Architecture

The localization pipeline splits responsibilities between two dedicated files in the scripts/ directory:

• scripts/readme_translations.py – Maintains the translation dictionary and defines translatable keys as string constants (e.g., HERO1, HERO2). This file contains the TRANSLATIONS nested dictionary where outer keys are ISO language codes ("es", "fr", "de") and inner maps associate English blocks with their target translations.
• scripts/build_readme_i18n.py – Implements the extraction and rendering engine. It parses the canonical README.md, identifies translatable regions using the spans() function, performs dictionary lookups against the translation tables, and writes localized versions to i18n/<lang>/README.md.

Translation Data Model in readme_translations.py

Block Keys as Translation Constants

Rather than using arbitrary identifiers, the script defines translation keys as exact English strings stored in module-level constants. For example:

HERO1 = "**84% of students already use AI tools…**"
HERO2 = "503 lessons. 20 phases. ~320 hours..."

These constants act as lookup keys in the TRANSLATIONS dictionary. Because each key matches the exact English text (with whitespace normalization via block_key()), any missing translation automatically falls back to the original English string, ensuring the build never breaks.

The TRANSLATIONS Dictionary Structure

The TRANSLATIONS constant is a nested dictionary mapping language codes to translation maps:

TRANSLATIONS = {
    "es": {
        HERO1: "**El 84% de los estudiantes ya usan herramientas de IA...",
        HERO2: "503 lecciones. 20 fases. ~320 horas...",
    },
    "fr": {
        HERO1: "**84% des étudiants utilisent déjà des outils d'IA...",
        HERO2: "503 leçons. 20 phases. ~320 heures...",
    }
}

Canonical Version Banner

Each translated README includes an optional README_NOTE constant—an HTML snippet inserted at the top of generated files indicating that the English version remains the authoritative source.

How build_readme_i18n.py Identifies Translatable Content

The extraction engine uses the spans() function to segment the canonical README into discrete, replaceable units while preserving code blocks and structural elements.

Span Detection Logic

The parser walks the README line-by-line in scripts/build_readme_i18n.py and categorizes content:

• Code fences are identified via the FENCE regex and excluded from translation to preserve syntax integrity.
• Headings match the HEADING regex and generate single-line spans categorized as "heading".
• Prose blocks are collected using the is_prose() helper to aggregate consecutive lines of narrative text into maximal runs marked as "prose" spans.

Key Normalization

The block_key() helper function normalizes extracted text by cleaning whitespace and line breaks to match the constants defined in readme_translations.py, enabling reliable dictionary lookups during the rendering phase.

Rendering and Link Localization

Bottom-Up Text Replacement

The render() function in scripts/build_readme_i18n.py performs substitutions in reverse index order to maintain valid string positions during modification:

for sp in sorted(spans(text), key=lambda s: s["start"], reverse=True):
    t = table.get(sp["key"])
    if not t: 
        continue
    replacement = [sp["prefix"] + ln for ln in t.split("\n")]
    lines[sp["start"]:sp["end"]] = replacement

If a translation is missing, the original English block remains untouched, ensuring graceful degradation and allowing incremental translation workflows.

Relative Link Adjustment

After text substitution, the localize_links() function rewrites repository-root-relative Markdown and HTML links to point two levels up (../../). This adjustment ensures that images, tables, and the language-selection bar resolve correctly from the generated file's location at i18n/<lang>/README.md.

Running the i18n Pipeline

The scripts/build_readme_i18n.py script supports three execution modes via command-line arguments:

# Generate all localized READMEs

python3 scripts/build_readme_i18n.py

# List all translatable block keys (useful for new translations)

python3 scripts/build_readme_i18n.py --dump

# Verify generated files match current sources (CI check)

python3 scripts/build_readme_i18n.py --check

In generate mode, the script iterates through each language entry in TRANSLATIONS, prepends the optional README_NOTE, and writes the final Markdown to i18n/<lang>/README.md.

In dump mode, the script prints every block_key discovered by spans() across the canonical README, providing contributors with the exact strings requiring translation.

In check mode, the script compares existing generated files against freshly computed outputs, reporting mismatches as stale translations that need regeneration before committing.

Adding a New Language to the Pipeline

Extending support to a new language requires three steps:

1. Update the translation table in scripts/readme_translations.py by adding a new language code entry (e.g., "no" for Norwegian) with translations for all defined keys such as HERO1, HERO2, and navigation headings.

2. Add the language code to the README's language selection bar in the canonical English file.

3. Run the generator to create i18n/<lang>/README.md automatically:

python3 scripts/build_readme_i18n.py

Because the system uses hand-authored translation tables, the landing page retains exact formatting control, and partial translations safely default to English for any missing keys.

Summary

• The readme-translations.py script stores hand-crafted translation tables using exact English text as lookup keys, enabling automatic fallback to English when translations are missing.
• build_readme_i18n.py scans the canonical README using the spans() function to identify headings and prose blocks while ignoring code fences marked by the FENCE regex.
• The render() function performs bottom-up text substitution to maintain index integrity, while localize_links() adjusts relative paths to ../../ for files in i18n/<lang>/ subdirectories.
• The pipeline supports three CLI modes: generation, key dumping for contributors, and staleness checking for continuous integration validation.
• New languages are added incrementally by extending the TRANSLATIONS dictionary without modifying the rendering logic, ensuring type-safe, community-driven localization.

Frequently Asked Questions

What happens if a translation is missing for a specific text block?

The renderer gracefully falls back to the original English text. Because translation keys in readme_translations.py are the exact English strings, a missing entry in the TRANSLATIONS dictionary causes the render() function to skip replacement and preserve the source text from the canonical README.

How does the script handle code blocks and technical syntax?

The spans() function in build_readme_i18n.py uses the FENCE regex pattern to identify and skip code blocks delimited by triple backticks. This ensures that shell commands, Python snippets, and configuration examples remain untranslated and syntactically intact during the localization process.

Can I test translations locally before submitting a pull request?

Yes. Run python3 scripts/build_readme_i18n.py --dump to extract all translatable keys and verify your translation table contains entries for each required constant. Then execute the standard generation command to produce the localized README locally in i18n/<your-lang>/README.md, allowing you to preview formatting and link resolution without affecting the canonical English source.

Why does the script rewrite relative links to ../../?

Generated READMEs live in i18n/<lang>/ subdirectories, two levels deeper than the repository root. The localize_links() function prepends ../../ to repository-root-relative URLs so that images, architecture diagrams, and license links resolve correctly when visitors view the localized landing pages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}