# AI Engineering i18n Translation Pipeline: Automated Machine-Translation on the translations Branch

> Explore the AI Engineering i18n translation pipeline automating machine-translation on the translations branch. Learn how English content becomes multilingual using Python and GitHub Actions.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-08-31

---

**The ai-engineering-from-scratch repository maintains a fully automated internationalization pipeline that machine-translates English lesson content into multiple languages using Python scripts and GitHub Actions, storing all generated files on the dedicated `translations` branch.**

The rohitg00/ai-engineering-from-scratch project implements a deterministic i18n translation pipeline that eliminates manual localization overhead. Rather than polluting the `main` branch with generated content, the system isolates all machine-translated curriculum files on a separate `translations` branch, enabling the static site generator to serve localized pages while preserving a clean source history.

## Architecture of the i18n Translation Pipeline

The pipeline follows a reproducible four-stage flow: **source extraction**, **machine-translation**, **file generation**, and **automated deployment**. Canonical English content lives in `phases/*/docs/en.md` on the `main` branch. When triggered, the system extracts markdown text—intentionally ignoring code blocks to prevent translation of syntax—processes it through a configurable translation service, and writes parallel directory structures under `i18n/<lang>/` and `phases/.../docs/<lang>.md` on the `translations` branch.

## The Translation Driver (scripts/translate_lessons.py)

The core orchestration logic resides in [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py). This script walks the lesson tree using `pathlib`, loads source files with the `frontmatter` library to preserve YAML metadata, and delegates translation to a generic wrapper function.

Key implementation details include:

- **LANGUAGES list**: A hardcoded array in [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py) defines supported locales: `["es", "fr", "de", "zh", "ja", "ko", "hi", "ar", "ru", "tr"]`.
- **`translate_text()` wrapper**: A helper function abstracts the machine-translation API, allowing seamless swapping between OpenAI ChatCompletion, Hugging Face MarianMT, or Google Translate.
- **Frontmatter preservation**: The script copies the original YAML frontmatter to translated files, ensuring the curriculum UI renders them without template modifications.

```python

# scripts/translate_lessons.py – main loop (simplified)

import os, pathlib, frontmatter
from translate import translate_text   # wrapper around the chosen MT service

LANGUAGES = ["es", "fr", "de", "zh", "ja", "ko", "hi", "ar", "ru", "tr"]

def translate_lesson(md_path: pathlib.Path):
    source = frontmatter.load(md_path)
    body = source.content
    for lang in LANGUAGES:
        translated = translate_text(body, target_lang=lang)
        target_dir = md_path.parent.parent / "i18n" / lang / md_path.parent.name
        target_dir.mkdir(parents=True, exist_ok=True)
        target_path = target_dir / "README.md"
        with target_path.open("w", encoding="utf‑8") as f:
            f.write("---\n")
            f.write(f"lang: {lang}\n")
            f.write("---\n\n")
            f.write(translated)

if __name__ == "__main__":
    for md in pathlib.Path("phases").rglob("docs/en.md"):
        translate_lesson(md)

```

## Machine-Translation Service Integration

By default, the pipeline utilizes the **OpenAI ChatCompletion** endpoint with a system prompt instructing the model to translate to the specified target language. The service layer is fully configurable via environment variables, decoupling the translation logic from specific vendors.

Configuration requirements:

- Set `OPENAI_API_KEY` in repository secrets for the default backend.
- Override the `translate_text()` implementation in [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py) to integrate alternative providers such as Hugging Face MarianMT or Google Cloud Translation.

The script specifically excludes fenced code blocks from the translation payload, ensuring that Python syntax, shell commands, and configuration examples remain intact across all language versions.

## CI/CD Automation with GitHub Actions

The [`.github/workflows/translate.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/translate.yml) workflow automates pipeline execution on every push to `main`. It ensures the `translations` branch remains synchronized with the latest English curriculum without manual intervention.

```yaml

# .github/workflows/translate.yml – CI trigger

name: Translate Lessons
on:
  push:
    branches: [main]
jobs:
  translate:
    runs-on: ubuntu‑latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Set up Python
        uses: actions/setup-python@v4
        with: {python-version: "3.11"}
      - name: Install deps
        run: pip install -r .github/translate-requirements.txt
      - name: Run translation script
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: python -m scripts.translate_lessons
      - name: Push translations
        uses: ad-m/github-push-action@v0.6.0
        with:
          branch: translations
          force: true
          github_token: ${{ secrets.GITHUB_TOKEN }}

```

The workflow installs Python 3.11, resolves dependencies from [`.github/translate-requirements.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/translate-requirements.txt) (which includes the `openai` client), executes the driver, and force-pushes the generated i18n files to the `translations` branch.

## Static Site Generation and Language Routing

When the website builds, [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) reads the language-specific [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) files from the `translations` branch. The generator constructs navigation menus by parsing the markdown links within `i18n/<lang>/README.md` files, which follow the same relative path patterns as the English source. This design allows the static site to serve localized lesson pages at predictable URLs without requiring duplicate template logic.

## Extending the Pipeline to New Languages

Adding support for additional locales requires minimal changes:

1. Create a new directory under `i18n/` (e.g., `i18n/xx/`).
2. Append the ISO language code to the `LANGUAGES` list in [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py).
3. Commit and push to `main`; the GitHub Actions workflow will generate the full translation set on the next run.

This modular approach ensures the i18n translation pipeline scales horizontally as the curriculum expands into new regions.

## Summary

- The `translations` branch isolates all machine-generated content from the `main` branch source files.
- [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py) extracts markdown from `phases/*/docs/en.md` while preserving code blocks and frontmatter.
- Translation occurs via a configurable `translate_text()` wrapper, defaulting to OpenAI ChatCompletion.
- [`.github/workflows/translate.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/translate.yml) triggers on every push to `main`, automating the full generation and deployment cycle.
- The static site generator consumes `i18n/<lang>/README.md` files to build language-specific navigation menus.

## Frequently Asked Questions

### What triggers the i18n translation pipeline?

A push event to the `main` branch triggers the [`.github/workflows/translate.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/translate.yml) GitHub Actions workflow. This ensures the `translations` branch always reflects the current state of the English curriculum without requiring manual execution of the Python scripts.

### How does the pipeline handle code blocks during machine-translation?

The [`translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/translate_lessons.py) script parses markdown to extract only prose content, deliberately excluding fenced code blocks from the translation payload. This prevents corruption of Python syntax, YAML configurations, and shell commands, inserting them unchanged into the translated output files.

### Can I use a different machine-translation service instead of OpenAI?

Yes. The `translate_text()` function in [`scripts/translate_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/translate_lessons.py) serves as a pluggable abstraction layer. You can replace the default OpenAI ChatCompletion implementation with calls to Hugging Face MarianMT, Google Cloud Translation, or any other MT API by modifying this wrapper function and updating the environment variables in the GitHub Actions workflow.

### Why are translations stored on a separate branch instead of main?

The `translations` branch acts as a generated artifact store, similar to a `gh-pages` branch. This separation of concerns keeps the `main` branch history clean and reviewable, prevents merge conflicts in generated files, and allows the static site builder to pull localized content via a simple branch reference without cluttering the source repository with machine-generated markdown.