# How the Six-Volume Book Series Is Generated from Lesson Sources in AI Engineering from Scratch > Discover how the AI Engineering from Scratch book series is automatically generated. Learn about the Python pipeline that maps lessons, adds interactive links, and compiles EPUB/PDF outputs. - Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch) - Tags: how-to-guide - Published: 2026-09-04 --- **The six-volume *AI Engineering from Scratch* book series is generated by an automated Python pipeline that maps course phases to volumes, transforms individual lesson markdown files with interactive web links, and compiles them into EPUB and PDF formats using Pandoc.** The `rohitg00/ai-engineering-from-scratch` repository contains a sophisticated build system that converts modular lesson sources into a cohesive six-volume book series. This automated pipeline bridges the gap between living online course content and static publication formats. Understanding how the six-volume book series is generated from lesson sources reveals the architecture behind maintaining synchronized web and print educational materials. ## Volume Definition and Phase Mapping The pipeline begins with a static JSON configuration that establishes the structure of the six-volume book series. The file [`book/volumes.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/book/volumes.json) maps each volume to specific course phases, defining the organization of the entire curriculum. ```json { "volumes": [ { "slug": "foundations", "number": 1, "title": "Foundations", "phases": ["00-setup-and-tooling", "01-math-foundations", "02-ml-fundamentals"] }, { "slug": "production", "number": 6, "title": "Production", "phases": ["17-infrastructure-and-production", "18-ethics-safety-alignment", "19-capstone-projects"] } ] } ``` This configuration allows the build script to iterate through specific phases and aggregate the correct lesson sources for each volume. The **phase-based organization** ensures that the linear progression of the course maintains logical continuity across the six separate books. ## Lesson Discovery and Directory Scanning For each phase defined in the JSON mapping, the build system scans the filesystem to locate actual lesson content. The `lesson_dirs()` function in [`scripts/build_book.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_book.py) (lines 45-53) searches the `phases//` directory structure for folders containing a [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file. This discovery mechanism treats the repository's folder structure as the single source of truth. By scanning for [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) within each phase directory, the pipeline automatically includes new lessons as they are added to the course without requiring manual updates to the build configuration. ## Transforming Lesson Content for Print Once discovered, each lesson markdown undergoes transformation via the `transform_lesson()` function in [`scripts/build_book.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_book.py) (lines 9-80). This critical step adapts web-native content for static book formats while preserving references to interactive elements. ### URL Generation and Web Integration The `urls_for()` function generates three essential URLs for every lesson: ```python def urls_for(phase, lesson): rel = f"phases/{phase}/{lesson}" return { "web": f"{SITE}/lesson?path={rel}", "code": f"{REPO}/tree/main/{rel}/code", "repo": f"{REPO}/tree/main/{rel}", } ``` These URLs enable the **"continue online" pattern**—every printed chapter directs readers back to the live web edition for interactive features, updated code, and browser-graded quizzes. ### Markdown Transformation Pipeline The `transform_lesson()` function performs several specific content conversions: - **Figure blocks**: Converts ` ```figure` fenced blocks into boxed notices linking to interactive figures on the web edition - **Mermaid diagrams**: Renders ` ```mermaid` blocks into SVG using the Mermaid CLI when available, or falls back to web-edition pointers - **Section replacement**: Substitutes "## Ship It" and "## Exercises" sections with "continue-online" boxes pointing to starter code and solutions - **Asset path rewriting**: Adjusts image paths so Pandoc can properly embed assets during compilation - **Continuation boxes**: Appends a standardized box containing web links, code repositories, and quiz locations to every lesson The **continue-online box** implementation ensures readers always have access to the most current version of the material: ```python def continue_box(u, has_quiz): lines = [ "**Continue online.** The living edition of this chapter has more than the page can hold:", "", f"- Animated, interactive figures and the web text: <{u['web']}>", f"- Runnable code for every step: <{u['code']}>", ] if has_quiz: lines.append(f"- The chapter quiz, graded in the browser: <{u['web']}>") lines += [ "", "The repository moves faster than any printing. When the book and the repo disagree, trust the repo.", ] return fenced_div("continue-online", *lines) ``` ## Volume Assembly and Rendering The `assemble()` function in [`scripts/build_book.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_book.py) (lines 89-106) constructs the final markdown for each volume by concatenating transformed lessons. This process includes: 1. Inserting "How to use this volume" front-matter via `how_to_use()` 2. Adding "Part I/II/..." headings for every phase transition 3. Counting chapters and words for metadata tracking After assembly, the `render()` function (lines 22-44) writes a YAML metadata header and invokes **Pandoc** to generate the EPUB format. When the `--pdf` flag is provided and the language is left-to-right, the pipeline also produces PDF output using **XeLaTeX** with DejaVu fonts. You can execute the build process using the command-line interface: ```bash # Build every volume as EPUB (default) python3 scripts/build_book.py # Build only the "language" volume python3 scripts/build_book.py --volume language # Build EPUB + PDF for all volumes (requires xelatex and DejaVu fonts) python3 scripts/build_book.py --pdf ``` ## Continuous Integration and Automated Publishing A GitHub Actions workflow defined in [`.github/workflows/build-book.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/build-book.yml) automates the entire pipeline. The CI triggers on every push to the `phases/` directory tree, ensuring the six-volume book series remains synchronized with the latest lesson sources. The workflow publishes generated EPUBs to `dist/book/` and creates PDF artifacts for release tags. This automation maintains the principle that while the book serves as a snapshot of the course, the repository remains the authoritative source. ## Summary - **JSON configuration** in [`book/volumes.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/book/volumes.json) maps specific course phases to each of the six volumes, establishing the book structure. - **Automated discovery** via `lesson_dirs()` scans phase directories for [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) files, dynamically including new lessons. - **Content transformation** through `transform_lesson()` converts web-native markdown to print-ready formats while embedding "continue online" references. - **Pandoc compilation** generates EPUB output with optional PDF rendering via XeLaTeX for supported languages. - **GitHub Actions** provides continuous integration, automatically rebuilding the book series when lesson sources change. ## Frequently Asked Questions ### What file formats does the six-volume book series generate? The pipeline produces **EPUB** as the default format for all volumes. When the `--pdf` flag is specified and the language direction is left-to-right, the system also generates **PDF** files using XeLaTeX. Final artifacts are stored in `dist/book/` with filenames following the pattern `aiefs-volX-.epub`. ### How does the build system handle interactive figures and Mermaid diagrams? The `transform_lesson()` function processes ` ```figure` blocks by converting them into boxed notices that link to the interactive web edition. For ` ```mermaid` blocks, the pipeline attempts SVG rendering via the Mermaid CLI; if unavailable, it falls back to web-edition pointers. This ensures static books reference dynamic content without breaking the reading experience. ### Can I build individual volumes instead of the entire series? Yes. The [`scripts/build_book.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_book.py) script accepts a `--volume` argument that accepts a volume slug (such as `language` or `foundations`). Running `python3 scripts/build_book.py --volume language` will process only the phases mapped to that specific volume in [`book/volumes.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/book/volumes.json), significantly reducing build time during development. ### Where does the pipeline store intermediate and final files? Intermediate markdown assemblies are written to `book/_build/.md` during the `assemble()` phase. Final EPUB artifacts are published to `dist/book/aiefs-volX-.epub`, with PDF variants stored in the same directory when generated. The [`.github/workflows/build-book.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/build-book.yml) CI configuration manages these paths during automated builds.