# What Does `scripts/audit_lessons.py` Validate? Curriculum Compliance Rules Explained

> Learn how scripts/audit_lessons.py ensures curriculum compliance in ai-engineering-from-scratch by validating ten rules for lessons. Ensure directory naming, docs, quizzes, and links are correct before CI.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-09-02

---

**The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) script validates that every lesson in the `rohitg00/ai-engineering-from-scratch` repository conforms to ten strict structural, content, and schema compliance rules (L001-L010), ensuring directory naming consistency, documentation quality, quiz integrity, and internal link resolution before code reaches the CI pipeline.**

Maintaining consistency across hundreds of lessons in an open-source curriculum requires automated enforcement. The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) utility acts as a lint-style validator that audits the entire `rohitg00/ai-engineering-from-scratch` repository against canonical formatting standards. It guarantees that every lesson directory contains properly structured documentation, functional code examples, valid assessment files, and resolvable internal references.

## Directory and File Structure Validation (L001-L005)

The validator first ensures that lesson containers and mandatory files follow the repository's organizational schema.

### Lesson Naming Conventions (L001)

Every lesson directory must match the `NN-slug` pattern (e.g., `00-introduction`, `12-transformers`). In [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), the `LESSON_DIR_RE` regex pattern (lines 25-31) defines the allowed format, while the `check_lesson_dir_pattern()` function (lines 85-94) executes this validation against each lesson path. Directories that fail to use the two-digit prefix and kebab-case suffix trigger an immediate compliance violation.

### Documentation Standards (L002-L004)

Each lesson must include a [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file meeting three specific criteria enforced by the `check_docs_en_md()` function:

- **L002**: The file must exist and be valid UTF-8. The validator attempts to decode the file and catches `UnicodeDecodeError` to detect encoding issues (lines 98-106).
- **L003**: Documentation must be substantive—minimum 200 bytes. This check at lines 107-113 prevents placeholder or empty content from entering the curriculum.
- **L004**: The markdown must contain a top-level H1 heading (`# ...`). A regex test at lines 114-115 verifies this requirement, ensuring consistent document hierarchy for rendering.

### Code Directory Integrity (L005)

Lessons cannot submit empty `code/` directories. The `check_code_main()` function (lines 124-126) scans each lesson's code folder to confirm it contains at least one non-ignored source or configuration file (with validation logic defined at lines 19-26). This prevents incomplete lessons from passing review.

## Quiz Schema and Content Validation (L006-L009)

The validator enforces strict data integrity rules on assessment files using the `check_quiz()` function.

### JSON Structure and Legacy Detection (L006-L007)

- **L006**: The [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file must parse as valid JSON and contain a non-empty `questions` array. Each question must include all required canonical keys: `stage`, `question`, `options`, `correct`, and `explanation` (lines 30-54, 56-71).
- **L007**: The script detects legacy quiz schemas that use deprecated keys (`q`, `choices`, `answer`) and flags them for migration to the current format (lines 57-65).

### Answer Key Integrity (L008-L009)

The validator ensures that quizzes are logically consistent and answerable:

- **L008**: Each question's `options` array must contain between 2 and 6 entries (lines 76-84).
- **L009**: The `correct` field must be an integer index that points to a valid position within the `options` array (lines 86-93).

## Internal Link Validation (L010)

Broken cross-references disrupt the learning experience. The `check_internal_links()` function (lines 96-112) scans all markdown files for internal references using the pattern `[text](href)`. It resolves both relative and absolute paths to verify that every link targets an existing file or directory within the repository. This check can also be run independently using the companion [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py) utility.

## Running the Curriculum Audit

Execute the validator from the repository root to scan all lesson directories:

```bash
python scripts/audit_lessons.py

```

The script aggregates all findings into an `Audit` object and returns exit code `0` for a clean repository or `1` when violations exist, enabling CI pipelines to fail fast.

Generate a machine-readable JSON report for programmatic processing:

```bash
python scripts/audit_lessons.py --json > audit_report.json

```

The JSON payload includes `lessons_checked` and an array of issue objects with `rule`, `lesson`, `file`, and `message` fields.

Limit validation to a specific curriculum phase:

```bash
python scripts/audit_lessons.py --phase 5

```

Enable strict mode to treat all warnings as errors:

```bash
python scripts/audit_lessons.py --strict

```

## Summary

- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** enforces ten compliance rules (L001-L010) across the entire `rohitg00/ai-engineering-from-scratch` curriculum.
- **Structural checks** validate the `NN-slug` directory naming pattern, mandatory [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) presence, 200-byte minimum documentation size, required H1 headings, and non-empty `code/` folders.
- **Quiz validation** ensures JSON parses correctly, uses current schema keys (not legacy `q`/`choices`), provides 2-6 options per question, and indexes correct answers within valid bounds.
- **Link integrity** verification confirms all internal markdown references resolve to existing repository paths via `check_internal_links()`.
- The script supports CI integration through exit codes (`0` or `1`) and provides JSON output for automated reporting pipelines.

## Frequently Asked Questions

### What happens if a lesson directory fails the L001 naming check?

The audit reports a structural violation when a directory does not match the `NN-slug` pattern defined by `LESSON_DIR_RE` in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py). You must rename the folder to use a two-digit numeric prefix followed by a kebab-case descriptor (e.g., `03-nlp-basics`) before the CI pipeline will pass validation.

### Can I validate quiz files separately from the full audit?

While [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) runs all ten checks together, the quiz-specific logic is encapsulated in the `check_quiz()` function (lines 30-93). For isolated link validation (rule L010), use the companion [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py) script which implements the link resolution checks independently.

### Why does the validator require documentation to be at least 200 bytes?

Rule L003 enforces substantive lesson descriptions by checking file size in `check_docs_en_md()` (lines 107-113). This minimum threshold prevents placeholder or trivial documentation files from entering the curriculum, ensuring every lesson provides meaningful explanatory content for learners.

### How does the script detect obsolete quiz formats?

Rule L007 scans for legacy schema keys (`q`, `choices`, `answer`) during [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) parsing (lines 57-65). When detected, the audit flags these structures so maintainers can migrate them to the current canonical format using `stage`, `question`, `options`, `correct`, and `explanation` keys.