# What Validation Rules Does the audit_lessons.py Script Apply to the 511 Lessons?

> Discover the 10 validation rules the audit_lessons.py script enforces for AI Engineering from Scratch lessons. Learn about naming conventions, documentation, code, quizzes, and links.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-08-29

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script enforces 10 canonical linting rules (L001–L010) across every lesson directory in the AI Engineering from Scratch curriculum, validating directory naming conventions, documentation completeness, code presence, quiz schema integrity, and internal link resolution.**

The `rohitg00/ai-engineering-from-scratch` repository structures its curriculum into 511 lessons organized across multiple phases. To ensure consistent quality and structural integrity, the repository includes [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), a purpose-built linting engine that traverses the `phases/` hierarchy and validates each lesson against strict canonical requirements.

## The 10 Canonical Validation Rules

The validator assigns a unique rule code (L001 through L010) to every class of violation. Each check targets a specific structural or content invariant required by the curriculum infrastructure.

### L001 – Directory Naming Convention

Rules require every lesson directory name to match the pattern `NN-slug`: a two-digit phase number followed by a kebab-case slug. This guarantees a predictable, sortable structure across the entire curriculum. The check is implemented via the `LESSON_DIR_RE` regular expression in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) at lines 25–94.

### L002 – Required Documentation File

Every lesson must contain a human-readable description at [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md). The script verifies file existence through the `check_docs_en_md` test at lines 98–101.

### L003 – Minimum Documentation Size

To prevent empty or placeholder documentation, [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must exceed `MIN_DOC_BYTES` (200 bytes). This size check occurs at lines 107–113.

### L004 – H1 Heading Presence

The validator ensures [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) contains a top-level H1 heading (`# …`) using a dedicated regex. This guarantees the document follows Markdown conventions and remains discoverable by the site generator. The H1 regex logic resides at lines 28–15.

### L005 – Non-Empty Code Directory

Lessons must ship runnable source or configuration files. The script inspects the `code/` directory, ignoring harmless entries defined in `CODE_IGNORED_NAMES`, and flags truly empty directories at lines 32–26.

### L006 – Valid Quiz JSON Structure

The [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file must contain valid JSON with a non-empty `questions` array (or top-level list), where each question is an object. This guarantees assessment data can be parsed by the curriculum website and CI pipelines. The JSON loading and shape validation occurs at lines 130–146.

### L007 – Legacy Quiz Schema Detection

The script detects deprecated schema keys (`q`, `choices`, `answer`) and forces migration to the canonical schema using `stage`, `question`, `options`, `correct`, and `explanation`. Legacy-key detection runs at lines 157–165.

### L008 – Multiple Choice Option Limits

For valid multiple-choice formatting, each question’s `options` list must contain between `MIN_OPTIONS` (2) and `MAX_OPTIONS` (6) entries. This range check appears at lines 176–184.

### L009 – Correct Answer Index Validation

The `correct` value must be an integer index falling within the bounds of the `options` array. This prevents out-of-bounds answers that would break the quiz UI. The validation logic is implemented at lines 186–194.

### L010 – Internal Link Integrity

All internal Markdown links (relative or root-relative) in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must resolve to existing files or directories. This stops broken intra-repo hyperlinks that would produce 404s on the website. Link resolution logic executes at lines 196–112.

## How the Script Works

The validation engine operates through a four-stage pipeline defined in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py):

1.  **Phase and Lesson Discovery** – The `iter_lesson_dirs` function enumerates every `phases/*/*` folder, optionally filtered by the `--phase` argument.
2.  **Per-Lesson Auditing** – The `audit_lesson` function increments the lesson counter and executes the ten checks in sequence.
3.  **Issue Collection** – The `Audit.add` method stores each violation as an `Issue` object containing the rule code, lesson path, file, and message.
4.  **Reporting** – If invoked with `--json`, the script outputs a machine-readable JSON payload; otherwise, it renders a human-readable summary via `render_report`.

## Running the Validator

The script supports both interactive and CI-friendly execution modes. It returns exit code `0` for a clean repository and `1` if any issues are found (`return 1 if audit.issues else 0`).

Validate the entire curriculum:

```bash
python scripts/audit_lessons.py

```

Restrict validation to a specific phase:

```bash
python scripts/audit_lessons.py --phase 3

```

Generate a machine-readable JSON report for CI pipelines:

```bash
python scripts/audit_lessons.py --json > audit-report.json

```

Typical human-readable output:

```

audit_lessons.py — 45 lesson(s) checked, 3 issue(s)

  [L005] phases/02-foundations/lesson-01/code: code/ is empty (no source or config files)
  [L009] phases/04-advanced/lesson-12/quiz.json: question[2] correct=5 not a valid index in options[0..3]

Summary by rule:
  L005: 1
  L009: 1

```

## Summary

-   The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script serves as the canonical linting engine for the `rohitg00/ai-engineering-from-scratch` repository.
-   Ten distinct rules (L001–L010) enforce directory naming, documentation standards, code presence, and quiz schema integrity.
-   Each violation is recorded with a specific rule code, enabling precise debugging and automated CI filtering.
-   The script supports JSON output and phase-specific filtering for scalable validation workflows.
-   Exit codes follow standard conventions: `0` indicates success, `1` indicates one or more violations.

## Frequently Asked Questions

### How do I run the validator on only one phase of the curriculum?

Use the `--phase` argument followed by the two-digit phase number. For example, `python scripts/audit_lessons.py --phase 3` restricts validation to phase 03.

### What exit code does the script return when violations are found?

The script returns exit code `1` if any issues are detected, and `0` if the repository passes all checks. This behavior is implemented explicitly as `return 1 if audit.issues else 0`, making it suitable for CI pipeline gates.

### What is the minimum documentation size enforced by the validator?

The `MIN_DOC_BYTES` constant is set to **200 bytes**. Any [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file smaller than this threshold triggers an L003 violation, preventing empty or placeholder documentation from passing review.

### How does the validator detect broken internal links?

Rule L010 parses all Markdown links in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) that use relative or root-relative paths. It attempts to resolve each path against the repository structure; if the target file or directory does not exist, the script records a violation at lines 196–112.