How the Translation Workflow Keeps i18n README Files in Sync

The rohitg00/ai-engineering-from-scratch repository uses a deterministic, structure-preserving pipeline that extracts translatable spans from the canonical English README, maps them to hand-authored translations in a Python dictionary, and enforces strict synchronization through a CI check that fails when i18n files are stale.

Maintaining multilingual documentation accuracy is difficult when source content evolves frequently. The ai-engineering-from-scratch project solves this by treating README.md as the single source of truth and using a structure-preserving translation workflow that keeps i18n README files in sync without manual duplication. By deriving translation keys directly from exact English text blocks, the system automatically detects when content becomes outdated and blocks integration until updates are applied.

How the Translation Pipeline Works

Extracting Translatable Spans with spans()

The generator script scripts/build_readme_i18n.py begins by parsing the canonical English README with the spans() function. According to the ai-engineering-from-scratch source code, this identifies contiguous runs of headings, prose paragraphs, and blockquotes while excluding structural elements.

# From scripts/build_readme_i18n.py

def spans(text):
    # Detects heading, prose, and blockquote-prose runs

    ...

Non-prose elements—including tables, code fences, HTML blocks, badges, and links—are left untouched. This guarantees that the lesson table structure and layout remain identical across all language variants.

Mapping Translations in TRANSLATIONS

All localized content lives in scripts/readme_translations.py inside the TRANSLATIONS dictionary. The dictionary uses the exact normalized English text blocks as keys, mapping them to their localized equivalents.

Because keys are derived from raw English strings, any modification to the source text immediately invalidates the associated translation. This prevents silent staleness and forces explicit updates.

Rendering Content and Localizing Links

The render() function walks the extracted spans and substitutes English blocks with their translations where available. Blocks without translations retain their original English content. After substitution, localize_links() performs a second pass to adjust relative URLs and image paths so they resolve correctly from the i18n/<lang>/ directory depth.

# From scripts/build_readme_i18n.py

body = localize_links(render(text, lang, TRANSLATIONS))

Adding the Canonical Language Banner

Before writing output, the script prepends a language-specific notice defined in readme_translations.py as README_NOTE. This banner informs readers that the English version is the authoritative source.

Writing Output to i18n/<lang>/README.md

The script finalizes by writing the generated content to the appropriate language directory:

dst.parent.mkdir(parents=True, exist_ok=True)
dst.write_text(content, encoding="utf-8")

CI Enforcement and Stale Detection

Synchronization is enforced by a GitHub Actions job (triggered via the site/build.js → site/data.js pipeline) that runs the script with the --check flag. In this mode, the generator compares freshly rendered content against the committed files without modifying them.

If any language is out of date, the script exits with a non-zero status:

python3 scripts/build_readme_i18n.py --check

# Prints stale languages and exits with status 1 if mismatches exist

This failure blocks pull request merges, ensuring that i18n README files cannot drift from the canonical source.

Local Development and Testing

Generating Translations Locally

To update all i18n files after editing the English README, execute:

python3 scripts/build_readme_i18n.py

This regenerates every i18n/<lang>/README.md file. Commit these changes alongside your English edits to pass CI validation.

Unit Test Coverage

The test suite in scripts/test_translate_workflow.py validates the rendering logic and --check behavior. It renders sample English blocks, verifies translation key existence, and confirms that the stale-detection mechanism correctly identifies outdated files.

Summary

• Span extraction: The spans() function in scripts/build_readme_i18n.py isolates prose, headings, and blockquotes while preserving tables and code fences.
• Deterministic keys: The TRANSLATIONS dictionary in scripts/readme_translations.py uses exact English text as keys, automatically invalidating stale mappings when source content changes.
• Link adjustment: The localize_links() function rewrites relative paths to account for the deeper i18n/<lang>/ directory structure.
• CI gatekeeping: The --check flag enables the GitHub Actions workflow to fail builds when translations are out of sync, preventing stale documentation merges.
• Round-trip validation: The render() function preserves English content for untranslated blocks and maintains deterministic output, validated by scripts/test_translate_workflow.py.

Frequently Asked Questions

How does the workflow detect when a translation needs updating?

The system uses the exact English text block as the dictionary key in TRANSLATIONS. When you modify the source README, the spans() function extracts a new string that no longer matches the stored key, causing the render() output to differ from the committed file. The CI --check flag catches this mismatch and fails the build.

Why are tables and code blocks excluded from the translation process?

Structural elements like tables and code fences are deliberately excluded by the spans() function to prevent layout corruption. By translating only prose, headings, and blockquotes, the workflow ensures that complex markdown structures remain syntactically valid and visually consistent across all languages.

What is the purpose of the --check flag in CI?

The --check flag executes a dry-run that compares the generated content against the currently committed files in i18n/<lang>/README.md. If discrepancies exist, the script exits with status 1, causing the GitHub Actions job to fail and block the merge until the translations are regenerated and committed.

Can translators modify the generated README files directly?

No. The i18n/<lang>/README.md files are generated artifacts. All translation work must occur in scripts/readme_translations.py. Direct edits to the generated files would be overwritten the next time scripts/build_readme_i18n.py runs and would cause the deterministic check in CI to fail.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}