# What Invariant Checks Does audit_lessons.py Perform? Complete Guide to the AI Engineering Curriculum Auditor

> Explore the ten invariant checks L001–L010 performed by audit_lessons.py. Ensure your AI engineering lessons meet standards for naming, docs, code, quizzes, and links.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-13

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script enforces ten strict invariant checks (L001–L010) on every lesson directory, validating naming conventions, documentation standards, code presence, quiz schema integrity, and internal link resolution.**

The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) file in the `rohitg00/ai-engineering-from-scratch` repository functions as the automated lint engine for the curriculum. It traverses the `phases/` directory tree to ensure each lesson folder adheres to strict pedagogical and structural standards before CI approval. Understanding these invariant checks helps contributors prepare compliant lessons that pass automated quality gates.

## The 10 Invariant Checks Explained

The script categorizes violations into ten distinct rules, each targeting a specific structural requirement. Below is the complete breakdown of what [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) validates in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py).

### L001: Lesson Directory Naming Pattern

The `check_lesson_dir_pattern` function (lines 85‑94) validates that every lesson directory matches the pattern `NN-slug`, where `NN` represents two digits and `slug` consists of lowercase alphanumerics and hyphens. For example, `01-intro-to-tensors` passes validation, while `1-intro` or `01_Intro` would trigger a violation.

### L002: Presence of docs/en.md

The `check_docs_en_md` function (lines 98‑101) ensures each lesson contains a [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file. This invariant guarantees that every lesson has a primary documentation entry point in English.

### L003: Minimum Documentation Size

Within the same `check_docs_en_md` function (lines 107‑113), the script verifies that [`en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/en.md) contains at least 200 bytes. This check prevents placeholder or empty documentation files from entering the curriculum.

### L004: Top-Level H1 Header

Also in `check_docs_en_md` (lines 114‑115), the audit confirms the markdown document starts with a proper ATX-style H1 header (`# Title`). Lessons missing this top-level heading fail the invariant.

### L005: Non-Empty Code Directory

The `check_code_main` function (lines 119‑126) inspects the `code/` directory to ensure it contains at least one source file, excluding ignored names. This invariant guarantees that every lesson includes implementable code examples.

### L006: Valid quiz.json Structure

The `check_quiz` function (lines 129‑152) validates that [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) is properly formatted JSON containing a non-empty `questions` list. Each question must include the canonical keys: `stage`, `question`, `options`, `correct`, and `explanation`.

### L007: No Legacy Quiz Schema

Within `check_quiz` (lines 157‑166), the script explicitly forbids legacy key names such as `q`, `choices`, or `answer`. This ensures all quizzes use the modern, standardized schema throughout the repository.

### L008: Options Count Bounds

The `check_quiz` function (lines 176‑184) enforces that each question's `options` list contains between 2 and 6 entries. Questions with fewer than 2 or more than 6 options violate this invariant.

### L009: Correct-Index Validity

In `check_quiz` (lines 186‑193), the audit verifies that the `correct` field is an integer pointing to a valid index within the `options` array. Out-of-bound indices or non-integer values trigger a rule violation.

### L010: Internal Markdown Link Resolution

The `check_internal_links` function (lines 196‑112) validates that every relative link inside [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) resolves to an existing file within the repository. Broken internal links prevent curriculum navigation and fail this check.

## How the Audit Orchestration Works

Understanding the execution flow reveals how these invariant checks integrate into the CI pipeline.

### Directory Enumeration

The `iter_lesson_dirs` function (lines 65‑82) walks the `phases/` tree, optionally filtering by a `--phase` argument. This generator yields each lesson directory path for individual inspection.

### Sequential Validation

For each lesson folder, the `audit_lesson` function (lines 144‑152) increments a counter and executes the ten checks in the order listed above. All violations are collected into an `Audit` object's `issues` list for batch reporting.

### Reporting and Exit Codes

The `render_report` function (lines 224‑272) produces either a human-readable summary or a machine-readable JSON dump when using the `--json` flag. The script exits with code 1 if any invariant check fails, which CI systems interpret as a merge-blocking failure.

## Running the Audit Locally

Execute the audit from the repository root to validate the entire curriculum:

```bash
python scripts/audit_lessons.py

```

Typical output shows the count of lessons checked and specific violations:

```text
audit_lessons.py — 215 lesson(s) checked, 3 issue(s)

  [L004] phases/03-ml-basics/01-linear-regression/docs/en.md: docs/en.md missing top-level H1
  [L008] phases/07-optimizers/02-sgd/quiz.json: question[2] options length must be 2..6 (got 1)

Summary by rule:
  L004: 1
  L008: 1

```

For programmatic integration or CI pipelines, output JSON:

```bash
python scripts/audit_lessons.py --json > audit_report.json

```

The JSON output follows this structure:

```json
{
  "lessons_checked": 215,
  "issues": [
    {
      "rule": "L004",
      "lesson": "phases/03-ml-basics/01-linear-regression",
      "file": "phases/03-ml-basics/01-linear-regression/docs/en.md",
      "message": "docs/en.md missing top-level H1"
    }
  ]
}

```

## Summary

- **Ten invariant checks (L001–L010)** enforce strict structural standards across all lesson directories in `rohitg00/ai-engineering-from-scratch`.
- **Directory naming, documentation presence, and size** are validated by `check_lesson_dir_pattern` and `check_docs_en_md`.
- **Code presence** is verified by `check_code_main`, ensuring every lesson includes implementation files.
- **Quiz integrity** is guarded by `check_quiz`, which validates schema, legacy key absence, option bounds, and correct answer indices.
- **Link resolution** is confirmed by `check_internal_links`, preventing broken navigation.
- The script exits with **code 1** on any violation, blocking CI merges until all invariants are satisfied.

## Frequently Asked Questions

### What happens when a lesson fails an invariant check?

The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script collects all violations in an `Audit` object and exits with status code 1. In CI pipelines, this non-zero exit code blocks pull request merges, enforcing that all ten invariant checks must pass before a lesson reaches the main branch.

### Can I audit only a specific phase instead of the entire curriculum?

Yes. The `iter_lesson_dirs` function supports a `--phase` argument. Running `python scripts/audit_lessons.py --phase 03-ml-basics` restricts the audit to lessons within that specific phase directory, speeding up validation during targeted development.

### What are the specific requirements for quiz.json validation?

The `check_quiz` function requires [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) to contain a valid JSON structure with a `questions` array. Each question must include the canonical keys `stage`, `question`, `options`, `correct`, and `explanation`. The `options` array must contain 2–6 items, and the `correct` value must be an integer index within the bounds of that array. Legacy keys like `q`, `choices`, or `answer` are explicitly prohibited.

### How does the script verify internal documentation links?

The `check_internal_links` function (lines 196‑112) parses [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) for relative markdown links and verifies that each target path exists within the repository filesystem. This ensures that cross-references between lessons remain functional and that no broken links enter the curriculum documentation.