How to Add a New Lesson Using scaffold-lesson.sh in AI Engineering from Scratch

Execute scripts/scaffold-lesson.sh <phase-directory> <lesson-slug> [optional-title] from the repository root to generate a standardized lesson skeleton with pre-populated documentation, directory hierarchy, and starter code files.

The ai-engineering-from-scratch curriculum (maintained at rohitg00/ai-engineering-from-scratch) enforces strict architectural conventions to support automated site generation and CI validation. The scaffold-lesson.sh script eliminates manual setup errors by programmatically creating the required folder structure and seeding docs/en.md from LESSON_TEMPLATE.md.

Script Location and Core Functionality

The scaffolding logic is implemented in scripts/scaffold-lesson.sh. When invoked, the script performs three atomic operations to ensure structural consistency:

1. Directory Initialization: Creates the mandatory four-folder layout under phases/<phase>/<lesson-slug>/: code/, notebook/, docs/, and outputs/.
2. Documentation Seeding: Copies LESSON_TEMPLATE.md into docs/en.md, preserving the required YAML front-matter (type, languages, prerequisites, time) defined in the repository's AGENTS.md policy file.
3. Code Stub Generation: Writes an empty code/main.py (or language-appropriate equivalent) with a standardized header comment pointing to the lesson documentation.

Command Syntax and Parameters

The script accepts three positional arguments with strict validation:

scripts/scaffold-lesson.sh <phase-dir> <lesson-slug> [title]

• <phase-dir>: The target phase directory under phases/. This directory must pre-exist; the script validates its presence to prevent orphaned lessons outside the curriculum hierarchy.
• <lesson-slug>: Must conform to the NN-kebab-case pattern, where NN is a two-digit zero-padded lesson number (e.g., 03-tokenizers). The script enforces this via regex to ensure proper lexical sorting.
• [title]: Optional human-readable title. If omitted, the script derives a title by converting hyphens to spaces and capitalizing each word in the slug.

Step-by-Step Workflow

Execute the Scaffolding Command

Run the script from the repository root:

scripts/scaffold-lesson.sh 05-nlp-foundations-to-advanced 03-tokenizers "Tokenizers from Scratch"

This creates phases/05-nlp-foundations-to-advanced/03-tokenizers/ with the complete subfolder hierarchy and templated files.

Configure Documentation Metadata

Edit phases/<phase>/<lesson>/docs/en.md to replace template placeholders:

• Set the type field to concept, implementation, or project.
• Define supported languages in the front-matter array (e.g., python, javascript).
• List prerequisites using slugs from previous lessons.
• Set the estimated time for completion.
• Fill the required sections: Problem Description, Core Concept, Step-by-Step Build Instructions, Usage Discussion, and Ship Artifact Description.

Implement Logic and Tests

Navigate to the generated code/ directory to develop the lesson:

1. Write the core implementation in main.py (or the appropriate language file).
2. Add unit tests under code/tests/ following the repository's testing conventions.
3. Validate locally before committing:

   cd phases/05-nlp-foundations-to-advanced/03-tokenizers/code
   python3 main.py && python3 -m unittest discover tests -v

Update Curriculum Indexes

The script does not modify navigation files automatically. You must manually append references to:

• The root README.md (master curriculum index)
• phases/<phase>/README.md (phase-specific index)
• ROADMAP.md (updating status from 🟡 Planned to 🟢 In Progress)

Generated Directory Structure

The scaffold creates a layout that matches the validation logic in scripts/audit_lessons.py:

phases/05-nlp-foundations-to-advanced/03-tokenizers/
├── code/
│   ├── main.py          # Empty stub with documentation header

│   └── tests/           # Directory for unit test modules

├── notebook/            # Jupyter notebooks or equivalent

├── docs/
│   └── en.md            # Populated from LESSON_TEMPLATE.md

└── outputs/             # Build artifacts, figures, and generated data

Safety Checks and Repository Guards

The script implements protective validation to maintain repository integrity:

• Phase Existence Verification: Aborts immediately if the specified phase directory does not exist under phases/, preventing lessons from being created in invalid locations.
• Collision Detection: Checks for existing lesson directories before writing anything, eliminating accidental overwrites of published content.
• Slug Format Enforcement: Validates against the pattern ^[0-9]{2}-[a-z0-9-]+$ to enforce consistent sorting and URL generation across the curriculum.

Integration with CI and Commit Standards

After completing content development, follow the one-commit-per-lesson rule specified in AGENTS.md:

git add phases/05-nlp-foundations-to-advanced/03-tokenizers README.md ROADMAP.md
git commit -m "feat(phase-05/03): add tokenizers-from-scratch lesson"

Upon opening a pull request, the CI pipeline executes scripts/audit_lessons.py, which verifies:

• Presence of required front-matter in docs/en.md
• Existence of test files in code/tests/
• Valid cross-references in the prerequisites field
• Structural compliance with the four-directory standard

Summary

• scaffold-lesson.sh automates lesson creation in rohitg00/ai-engineering-from-scratch, ensuring compliance with architectural standards defined in AGENTS.md.
• The script requires an existing phase directory and a NN-kebab-case slug format (e.g., 03-tokenizers) to maintain curriculum ordering.
• It generates four mandatory subdirectories (code/, notebook/, docs/, outputs/) and seeds docs/en.md from LESSON_TEMPLATE.md.
• Manual updates to README.md, phase indexes, and ROADMAP.md are required after scaffolding.
• The CI system validates new lessons using scripts/audit_lessons.py to ensure adherence to documentation and testing standards.

Frequently Asked Questions

What happens if I provide an invalid lesson slug?

The script validates the slug against the NN-kebab-case pattern (two digits, hyphen, lowercase alphanumeric characters) and exits with an error if the format does not match. This enforcement prevents sorting errors and broken URLs in the generated curriculum site.

Can I create a lesson in a phase that doesn't exist yet?

No. The script explicitly checks for the existence of the phase directory under phases/ and aborts if it is not found. You must first establish the phase directory structure manually before scaffolding lessons within it.

How does the script handle multiple programming languages?

While the scaffold defaults to creating code/main.py, the docs/en.md front-matter includes a languages metadata field where you can declare additional languages (e.g., rust, go). You must manually create corresponding source files (e.g., main.rs) in the code/ directory, as the script initializes only the primary Python stub.

Why must I update README files manually after running the script?

The scaffolding tool focuses on lesson-local file generation to avoid merge conflicts in shared index files. The script intentionally does not modify README.md or ROADMAP.md, allowing maintainers to curate the curriculum order and status independently of the lesson creation process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}