# How the Audit Script in rohitg00/ai-engineering-from-scratch Validates Lesson Structure Invariants

> Learn how the audit script in rohitg00/ai-engineering-from-scratch validates lesson structure invariants with its deterministic eight-step pipeline checking directory naming, docs, code, quizzes, and links.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-06-12

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script in the `rohitg00/ai-engineering-from-scratch` repository enforces structural invariants across 500+ lessons by running a deterministic eight-step pipeline that checks directory naming, documentation requirements, code presence, quiz schemas, and internal links, aggregating violations into rule-coded issues ranging from L001 to L010.**

The `rohitg00/ai-engineering-from-scratch` curriculum organizes over 500 lessons across multiple phases, requiring strict structural consistency to remain maintainable. The **[`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py)** script serves as the central lint-like validator that ensures every lesson directory adheres to naming conventions, contains required documentation with valid markdown, and follows quiz schema standards. By validating these lesson structure invariants automatically, the script guarantees that content contributors cannot accidentally merge malformed lessons into the main branch.

## Lesson Discovery and Naming Conventions

The validation process begins with directory discovery. Starting at the repository `ROOT` (defined at lines 22–27), the script traverses the `phases/` folder using `iter_lesson_dirs()` (lines 65–83) to identify directories matching the strict regex pattern `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`.

Each phase and lesson must follow the `NN-slug` naming convention. The `check_lesson_dir_pattern()` function (lines 85–94) validates this format, emitting **issue L001** and skipping any malformed directories to prevent cascading validation errors.

## Documentation Validation (docs/en.md)

Every lesson must contain a [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file meeting four strict criteria enforced by `check_docs_en_md()` (lines 97–116):

- **Presence**: The file must exist (**L002**)
- **Encoding**: Must be valid UTF-8 (**L002**)
- **Minimum size**: At least 200 bytes (**L003**)
- **Top-level heading**: Must contain a markdown H1 (`# …`) (**L004**)

The function returns the file text for subsequent link validation, ensuring downstream checks operate on verified content.

## Code and Quiz Schema Enforcement

### Non-empty Code Directories

If a `code/` directory exists, it must contain at least one substantive file. The `check_code_main()` function (lines 119–127) excludes ignored files like [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and `.gitkeep`, raising **issue L005** if the directory is empty or contains only placeholder files.

### Quiz.json Schema Validation

The [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) validator enforces a strict schema through `check_quiz()` (lines 129–194). The script parses the JSON and validates that the structure is either a list of questions or an object containing a `questions` array.

Each question must satisfy:
- **Canonical keys only**: Using `stage`, `question`, `options`, `correct`, `explanation`—legacy keys trigger **L007**
- **Required fields**: Missing keys generate **L006**
- **Options length**: Between 2 and 6 items (**L008**)
- **Valid correct index**: Must be an integer within the options array bounds (**L009**)

## Internal Link Resolution

After successfully reading [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md), the script verifies all markdown links using `check_internal_links()` (lines 196–212). The resolver ignores absolute URLs, mailto, and data URIs, while resolving relative paths against the document's parent directory or repository root (for `/` prefixed links). Missing targets generate **issue L010**, which is shared with the companion [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py) utility for dedicated intra-repo link validation.

## Issue Aggregation and Reporting

The `Audit` class (lines 38–63) maintains a collection of `Issue` objects tracking violations across the curriculum. The `add()` method normalizes all paths to repository-relative POSIX format, ensuring consistent reporting across operating systems.

The `render_report()` function (lines 25–42) outputs results either as JSON (with `--json` flag) or human-readable text. The `main()` entry point (lines 44–73) orchestrates the validation flow and exits with status code 1 when violations exist.

Run the audit from the repository root:

```bash
python scripts/audit_lessons.py        # Plain text report

python scripts/audit_lessons.py --json # JSON output

python scripts/audit_lessons.py --phase 5  # Validate only Phase 5

```

Sample output shows rule-coded violations:

```text
audit_lessons.py — 87 lesson(s) checked, 4 issue(s)

  [L004] phases/09-reinforcement-learning/01-q-learning/docs/en.md: docs/en.md missing top-level H1
  [L008] phases/12-transformers/03-multihead-attention/quiz.json: question[2] options length must be 2..6 (got 1)

Summary by rule:
  L004: 1
  L008: 1

```

## Summary

- The **[`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py)** script in `rohitg00/ai-engineering-from-scratch` validates lesson structure invariants through an eight-step deterministic pipeline defined in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py).
- **Directory naming** is enforced via regex patterns in `iter_lesson_dirs()` (lines 65–83) and `check_lesson_dir_pattern()` (lines 85–94), with violations marked as **L001**.
- **Documentation standards** require UTF-8 encoded [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) files exceeding 200 bytes with proper H1 headings (**L002**–**L004**).
- **Code directories** must contain substantive files (**L005**), while **quiz.json** must follow strict schema rules for questions, options, and correct indices (**L006**–**L009**).
- **Internal links** are resolved and validated to prevent broken references (**L010**).
- The script aggregates all violations into an `Audit` object and exits with status 1 when structural invariants are violated, preventing regression in the 500+ lesson curriculum.

## Frequently Asked Questions

### What triggers an L001 error in the audit script?

An **L001** error occurs when a lesson directory name fails to match the required `NN-slug` pattern (`^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`). The `check_lesson_dir_pattern()` function detects this mismatch and skips further validation for that directory to prevent invalid path processing.

### How does the audit script validate quiz.json files?

The `check_quiz()` function (lines 129–194) parses the JSON and validates that each question uses the canonical schema keys (`stage`, `question`, `options`, `correct`, `explanation`), contains all required fields, provides 2–6 options, and specifies a valid correct index. Violations are reported as **L006** through **L009**.

### Can the audit script check only specific phases?

Yes. Running `python scripts/audit_lessons.py --phase 5` limits validation to Phase 5 only. The script filters lessons during the discovery phase in `iter_lesson_dirs()` before applying any invariant checks, optimizing validation time for large repositories.

### What is the difference between issues L002 and L004?

**L002** covers file existence and UTF-8 encoding failures in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md), while **L004** specifically flags missing top-level H1 headings within that file. A lesson could pass encoding checks but still fail if it lacks the required markdown header structure.