Optimal Repository Structure for Managing Phases and Lessons in AI Engineering Curriculum

The optimal repository structure for managing phases and lessons organizes content under phases/NN-phase-slug/NN-lesson-slug/ with standardized subdirectories (docs/, code/, outputs/), enabling automated validation, lexical ordering, and static site generation.

The rohitg00/ai-engineering-from-scratch repository implements a self-documenting curriculum hierarchy defined in AGENTS.md that scales to hundreds of lessons while maintaining discoverability. This optimal repository structure for managing phases and lessons treats the filesystem as a single source of truth, where directory naming conventions and markdown links drive both content organization and automated site generation.

Phase-Lesson Hierarchy Overview

The curriculum follows a strict two-level hierarchy documented in the Repo layout section of AGENTS.md. Phases (phases/NN-phase-slug/) represent coherent topic groups (e.g., Math Foundations, Deep Learning Core), while lessons (phases/NN-phase-slug/NN-lesson-slug/) serve as atomic learning units. Each lesson follows a reproducible structure containing docs/ for narrative content, code/ for implementations, and outputs/ for generated artifacts.

This design guarantees that learners can navigate directly to any topic via predictable paths, while automated tools in scripts/ validate every lesson against the same structural rules.

Directory Structure and Naming Conventions

The repository root contains phases, metadata, and automation scripts:

phases/
  00-phase-slug/
    00-lesson-slug/
      docs/
        en.md                # lesson narrative with front-matter

      code/
        main.<lang>          # runnable implementation

        tests/               # unit tests (≥5)

      outputs/               # generated artifact (prompt/skill/agent/MCP)

README.md                     # public index; auto-synced lesson counts

ROADMAP.md                    # phase/lesson status matrix

glossary/
  terms.md                    # canonical definitions

site/
  build.js                    # generates site/data.js from README

  data.js                     # generated, rebuilt by CI

scripts/
  *.py                        # automation (audit, count-fix, etc.)

Lexical Ordering with NN-Prefixes

The naming convention (NN-<slug>) guarantees deterministic URL generation and navigation ordering. For example, phases/03-deep-learning-core/15-new-lesson/ naturally sorts before 16-optimization-basics, which the static site generator relies on when building the navigation menu referenced in README.md § The shape of a lesson.

Key Design Benefits

• Discoverability: Every lesson lives under a predictable path (phases/<phase>/<lesson>/), allowing learners to jump directly to topics or use auto-generated tables of contents.

• Automation: site/build.js parses only the markdown links in README.md to generate site/data.js. Adding a new lesson requires only a single line in README.md; CI updates the site automatically.

• Consistency: Uniform subfolders (docs/, code/, outputs/) and required front-matter blocks in docs/en.md enable scripts/audit_lessons.py to validate every lesson with identical rules.

• Extensibility: Phase and lesson slugs are free-form but prefixed with an index (00, 01, …). New phases or lessons can be inserted without breaking existing URLs.

• Separation of Concerns: The outputs/ directory stores only the reusable artifact (prompt, skill, agent, or MCP server) while code/ holds the build-it implementation. Learners see both the raw algorithm and the library version without mixing the two.

• Version Control Hygiene: AGENTS.md enforces one commit per lesson directory, making PRs granular and rollback trivial.

Workflow for Adding New Lessons

According to the New-lesson onboarding section in AGENTS.md, contributors follow a seven-step workflow:

1. Create the scaffold

mkdir -p phases/15-new-phase/01-first-lesson/{docs,code,outputs}

2. Write the narrative (docs/en.md) with required front-matter (title, type, languages, prerequisites, time).

3. Add the implementation (code/main.py or code/main.ts) and matching tests under code/tests/.

4. Generate the artifact (outputs/skill-first-lesson.md) that ships as the reusable component.

5. Register the lesson in the README by adding a table row linking to the lesson folder:

| 15 | [New Lesson Title](phases/15-new-phase/01-first-lesson/) | Build | Python |

6. Update ROADMAP.md to mark lesson status (WIP, ✅, ⏳).

7. Run local validation before pushing:

python3 scripts/audit_lessons.py      # checks layout, front-matter, tests

python3 scripts/check_readme_counts.py --fix   # updates lesson counts

Example: scaffolding a lesson

# Scaffold a new lesson inside Phase 03 – Deep Learning Core

mkdir -p phases/03-deep-learning-core/15-new-lesson/{docs,code,outputs}

# Write the front-matter (docs/en.md)

cat > phases/03-deep-learning-core/15-new-lesson/docs/en.md <<'EOF'

# New Lesson Title

> One-line hook describing the core idea.

**Type:** Build
**Languages:** Python
**Prerequisites:** 14-loss-functions
**Time:** ~30

## Learning Objectives

- Implement X from first principles
- Compare X with library Y
- Ship a reusable prompt
EOF

Automated Site Generation and Validation

The build pipeline relies on the site/build.js Node.js script, which scans markdown links in README.md (e.g., [Dev Environment](phases/00-setup-and-tooling/01-dev-environment/)) to assemble a hierarchical JSON (site/data.js). This JSON powers the web frontend and the "Contents" navigation shown in the README.

If a lesson link is missing from README.md, CI flags it as a common bug and the site falls back to plain text, breaking navigation. The scripts/audit_lessons.py validator ensures every lesson directory contains the required docs/en.md with valid front-matter, runnable code files, and at least five unit tests in code/tests/.

Summary

• The optimal repository structure places lessons under phases/NN-phase-slug/NN-lesson-slug/ to ensure lexical ordering and deterministic URLs.
• Each lesson must contain docs/ (narrative), code/ (implementation with tests), and outputs/ (artifacts).
• AGENTS.md defines the canonical layout contract, while README.md and ROADMAP.md track progress and generate the site navigation.
• Automation scripts (scripts/audit_lessons.py, scripts/check_readme_counts.py) enforce consistency without manual intervention.
• The site/build.js generator creates site/data.js by parsing README.md links, making the filesystem the single source of truth.

Frequently Asked Questions

Why use numeric prefixes for phase and lesson directories?

The NN- prefix (e.g., 03-deep-learning-core, 15-transformers) guarantees lexical ordering in file listings and GitHub's web interface. This deterministic ordering ensures that site/build.js generates navigation menus and URLs in the correct pedagogical sequence without complex sorting logic.

What belongs in the outputs/ directory versus code/?

The code/ directory contains the build-it implementation that learners write from scratch (e.g., main.py, tests/), while outputs/ stores the shipped, reusable artifact (prompts, skills, agents, or MCP servers). This separation ensures learners see both the raw algorithm construction and the production-ready library version without mixing the two concerns.

How does the site generator handle missing lessons?

site/build.js parses only the explicit markdown links in README.md to build site/data.js. If a lesson exists in the filesystem but lacks a corresponding link in README.md, CI treats this as a common bug and the navigation will fall back to plain text, breaking the hierarchical menu structure. The scripts/check_readme_counts.py tool detects and fixes these discrepancies automatically.

What validation runs before merging a new lesson?

The scripts/audit_lessons.py validator checks that every lesson directory contains docs/en.md with valid front-matter (title, type, languages, prerequisites, time), a runnable code file in code/, at least five unit tests in code/tests/, and a non-empty artifact in outputs/. Additionally, scripts/check_readme_counts.py --fix ensures the lesson table in README.md remains synchronized with the actual directory count.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}