# What Checks Does audit_lessons.py Perform on Lesson Directories?

> Discover the 10 validation rules audit_lessons.py checks on lesson directories, ensuring naming, documentation, code, quizzes, and links meet standards in ai-engineering-from-scratch.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-22

---

**TLDR:** The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script enforces ten validation rules (L001–L010) on lesson directories under `phases/`, checking folder naming conventions, documentation completeness, code presence, quiz schema integrity, and internal link validity.

In the `rohitg00/ai-engineering-from-scratch` repository, the [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script serves as the automated quality gate for the curriculum. Located at [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), this validator traverses the `phases/` directory tree to ensure every lesson adheres to strict structural and content standards before publication.

## Validation Architecture and Execution Flow

The script operates through a pipeline of discovery and verification functions orchestrated by `audit_lesson()`. The `iter_lesson_dirs()` generator yields every lesson folder under `phases/<phase-NN>/<lesson-NN-slug>/`, with optional filtering via the `--phase` argument. Each discovered lesson triggers a sequence of helper functions that call `audit.add()` to record specific rule violations as structured `Issue` objects.

By default, the validator outputs a human-readable report summarizing all detected issues by rule code. For CI/CD integration, the `--json` flag emits a machine-readable payload containing complete issue details.

## The 10 Directory Validation Rules (L001–L010)

The validator implements each check as a dedicated function, assigning a unique rule code to every violation type.

### L001: Directory Naming Convention

The `check_lesson_dir_pattern` function (line 85) validates that lesson folders follow the strict `NN-slug` pattern. This requires a two-digit phase number, followed by a hyphen, then a lowercase alphanumeric string that may contain hyphens but must end with an alphanumeric character.

### L002–L004: Documentation Standards

The `check_docs_en_md` function (lines 97, 107, 114) verifies three critical aspects of the English documentation file:

- **L002:** Confirms the presence of [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) and validates UTF-8 encoding.
- **L003:** Ensures the file exceeds 200 bytes to prevent placeholder documentation.
- **L004:** Validates that the markdown contains a top-level heading (`# ...`).

### L005: Code Directory Requirements

The `check_code_main` function (line 119) inspects the `code/` subdirectory to ensure it is not empty. The check ignores generic filenames and requires at least one actual source or configuration file to be present.

### L006–L009: Quiz Schema Validation

The `check_quiz` function (lines 129, 157, 176, 186) performs comprehensive validation of [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files:

- **L006:** Verifies valid JSON structure with a non-empty `questions` array (or a root-level list), ensuring each question object contains all required keys.
- **L007:** Detects usage of the deprecated legacy schema using fields `q`, `choices`, and `answer`.
- **L008:** Validates that each question's `options` array contains between 2 and 6 entries.
- **L009:** Confirms the `correct` field is an integer index pointing to a valid position within the `options` list.

### L010: Internal Link Integrity

The `check_internal_links` function (line 196) parses [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) to verify that all internal markdown links resolve to existing files or directories within the repository, preventing broken references in the curriculum.

## Running the Audit Script

Execute the validator from the repository root to check all lessons:

```bash
python scripts/audit_lessons.py

```

To validate only a specific phase:

```bash
python scripts/audit_lessons.py --phase 02

```

For machine-readable output suitable for automated reporting:

```bash
python scripts/audit_lessons.py --json > audit_report.json

```

The script produces output similar to the following when issues are detected:

```text
audit_lessons.py — 125 lesson(s) checked, 7 issue(s)

  [L002] phases/02-ml-basics/01-linear-regression/docs/en.md: missing docs/en.md
  [L005] phases/03-nn-basics/05-activation-functions/code: code/ is empty (no source or config files)
  [L009] phases/04-optimizers/02-sgd/quiz.json: question[3] correct=5 not a valid index in options[0..3]

Summary by rule:
  L002: 1
  L005: 1
  L009: 1

```

## Summary

The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script functions as the curriculum's structural enforcer, implementing ten specific validation rules:

- **L001** enforces the `NN-slug` directory naming convention.
- **L002–L004** ensure [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists, is sufficiently large, and contains a proper heading.
- **L005** verifies the `code/` directory contains actual source files.
- **L006–L009** validate [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) structure, schema compliance, option counts, and correct answer indices.
- **L010** guarantees all internal documentation links resolve correctly.

## Frequently Asked Questions

### What is the primary purpose of audit_lessons.py?

The script serves as a lint-like validator that prevents structural errors, missing documentation, and malformed quizzes from entering the published curriculum. It automates quality control by verifying that every lesson directory under `phases/` adheres to the repository's strict formatting and content standards.

### How does the script validate quiz.json files?

The `check_quiz` function validates JSON syntax, ensures the presence of a `questions` array, checks for deprecated legacy schema usage, verifies that each question contains 2–6 options, and confirms that the `correct` index points to a valid option. These checks prevent broken quiz functionality in the learning platform.

### Can I run checks on a specific subset of lessons?

Yes. The `iter_lesson_dirs()` function supports the `--phase` command-line argument, allowing you to filter validation to a specific phase directory. For example, `python scripts/audit_lessons.py --phase 02` validates only the lessons within `phases/02-ml-basics/`.

### Where are the validation rules implemented in the source code?

Each rule is implemented as a specific function in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py). Directory naming checks reside in `check_lesson_dir_pattern` (line 85), documentation checks in `check_docs_en_md` (lines 97–114), code requirements in `check_code_main` (line 119), quiz validation in `check_quiz` (lines 129–186), and link verification in `check_internal_links` (line 196).