What Invariant Checks Does audit_lessons.py Perform? Complete Guide to the AI Engineering Curriculum Auditor

The audit_lessons.py script enforces ten strict invariant checks (L001–L010) on every lesson directory, validating naming conventions, documentation standards, code presence, quiz schema integrity, and internal link resolution.

The audit_lessons.py file in the rohitg00/ai-engineering-from-scratch repository functions as the automated lint engine for the curriculum. It traverses the phases/ directory tree to ensure each lesson folder adheres to strict pedagogical and structural standards before CI approval. Understanding these invariant checks helps contributors prepare compliant lessons that pass automated quality gates.

The 10 Invariant Checks Explained

The script categorizes violations into ten distinct rules, each targeting a specific structural requirement. Below is the complete breakdown of what audit_lessons.py validates in scripts/audit_lessons.py.

L001: Lesson Directory Naming Pattern

The check_lesson_dir_pattern function (lines 85‑94) validates that every lesson directory matches the pattern NN-slug, where NN represents two digits and slug consists of lowercase alphanumerics and hyphens. For example, 01-intro-to-tensors passes validation, while 1-intro or 01_Intro would trigger a violation.

L002: Presence of docs/en.md

The check_docs_en_md function (lines 98‑101) ensures each lesson contains a docs/en.md file. This invariant guarantees that every lesson has a primary documentation entry point in English.

L003: Minimum Documentation Size

Within the same check_docs_en_md function (lines 107‑113), the script verifies that en.md contains at least 200 bytes. This check prevents placeholder or empty documentation files from entering the curriculum.

L004: Top-Level H1 Header

Also in check_docs_en_md (lines 114‑115), the audit confirms the markdown document starts with a proper ATX-style H1 header (# Title). Lessons missing this top-level heading fail the invariant.

L005: Non-Empty Code Directory

The check_code_main function (lines 119‑126) inspects the code/ directory to ensure it contains at least one source file, excluding ignored names. This invariant guarantees that every lesson includes implementable code examples.

L006: Valid quiz.json Structure

The check_quiz function (lines 129‑152) validates that quiz.json is properly formatted JSON containing a non-empty questions list. Each question must include the canonical keys: stage, question, options, correct, and explanation.

L007: No Legacy Quiz Schema

Within check_quiz (lines 157‑166), the script explicitly forbids legacy key names such as q, choices, or answer. This ensures all quizzes use the modern, standardized schema throughout the repository.

L008: Options Count Bounds

The check_quiz function (lines 176‑184) enforces that each question's options list contains between 2 and 6 entries. Questions with fewer than 2 or more than 6 options violate this invariant.

L009: Correct-Index Validity

In check_quiz (lines 186‑193), the audit verifies that the correct field is an integer pointing to a valid index within the options array. Out-of-bound indices or non-integer values trigger a rule violation.

L010: Internal Markdown Link Resolution

The check_internal_links function (lines 196‑112) validates that every relative link inside docs/en.md resolves to an existing file within the repository. Broken internal links prevent curriculum navigation and fail this check.

How the Audit Orchestration Works

Understanding the execution flow reveals how these invariant checks integrate into the CI pipeline.

Directory Enumeration

The iter_lesson_dirs function (lines 65‑82) walks the phases/ tree, optionally filtering by a --phase argument. This generator yields each lesson directory path for individual inspection.

Sequential Validation

For each lesson folder, the audit_lesson function (lines 144‑152) increments a counter and executes the ten checks in the order listed above. All violations are collected into an Audit object's issues list for batch reporting.

Reporting and Exit Codes

The render_report function (lines 224‑272) produces either a human-readable summary or a machine-readable JSON dump when using the --json flag. The script exits with code 1 if any invariant check fails, which CI systems interpret as a merge-blocking failure.

Running the Audit Locally

Execute the audit from the repository root to validate the entire curriculum:

python scripts/audit_lessons.py

Typical output shows the count of lessons checked and specific violations:

audit_lessons.py — 215 lesson(s) checked, 3 issue(s)

  [L004] phases/03-ml-basics/01-linear-regression/docs/en.md: docs/en.md missing top-level H1
  [L008] phases/07-optimizers/02-sgd/quiz.json: question[2] options length must be 2..6 (got 1)

Summary by rule:
  L004: 1
  L008: 1

For programmatic integration or CI pipelines, output JSON:

python scripts/audit_lessons.py --json > audit_report.json

The JSON output follows this structure:

{
  "lessons_checked": 215,
  "issues": [
    {
      "rule": "L004",
      "lesson": "phases/03-ml-basics/01-linear-regression",
      "file": "phases/03-ml-basics/01-linear-regression/docs/en.md",
      "message": "docs/en.md missing top-level H1"
    }
  ]
}

Summary

• Ten invariant checks (L001–L010) enforce strict structural standards across all lesson directories in rohitg00/ai-engineering-from-scratch.
• Directory naming, documentation presence, and size are validated by check_lesson_dir_pattern and check_docs_en_md.
• Code presence is verified by check_code_main, ensuring every lesson includes implementation files.
• Quiz integrity is guarded by check_quiz, which validates schema, legacy key absence, option bounds, and correct answer indices.
• Link resolution is confirmed by check_internal_links, preventing broken navigation.
• The script exits with code 1 on any violation, blocking CI merges until all invariants are satisfied.

Frequently Asked Questions

What happens when a lesson fails an invariant check?

The audit_lessons.py script collects all violations in an Audit object and exits with status code 1. In CI pipelines, this non-zero exit code blocks pull request merges, enforcing that all ten invariant checks must pass before a lesson reaches the main branch.

Can I audit only a specific phase instead of the entire curriculum?

Yes. The iter_lesson_dirs function supports a --phase argument. Running python scripts/audit_lessons.py --phase 03-ml-basics restricts the audit to lessons within that specific phase directory, speeding up validation during targeted development.

What are the specific requirements for quiz.json validation?

The check_quiz function requires quiz.json to contain a valid JSON structure with a questions array. Each question must include the canonical keys stage, question, options, correct, and explanation. The options array must contain 2–6 items, and the correct value must be an integer index within the bounds of that array. Legacy keys like q, choices, or answer are explicitly prohibited.

How does the script verify internal documentation links?

The check_internal_links function (lines 196‑112) parses docs/en.md for relative markdown links and verifies that each target path exists within the repository filesystem. This ensures that cross-references between lessons remain functional and that no broken links enter the curriculum documentation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}