What Checks Does audit_lessons.py Perform on Lesson Directories?

TLDR: The audit_lessons.py script enforces ten validation rules (L001–L010) on lesson directories under phases/, checking folder naming conventions, documentation completeness, code presence, quiz schema integrity, and internal link validity.

In the rohitg00/ai-engineering-from-scratch repository, the audit_lessons.py script serves as the automated quality gate for the curriculum. Located at scripts/audit_lessons.py, this validator traverses the phases/ directory tree to ensure every lesson adheres to strict structural and content standards before publication.

Validation Architecture and Execution Flow

The script operates through a pipeline of discovery and verification functions orchestrated by audit_lesson(). The iter_lesson_dirs() generator yields every lesson folder under phases/<phase-NN>/<lesson-NN-slug>/, with optional filtering via the --phase argument. Each discovered lesson triggers a sequence of helper functions that call audit.add() to record specific rule violations as structured Issue objects.

By default, the validator outputs a human-readable report summarizing all detected issues by rule code. For CI/CD integration, the --json flag emits a machine-readable payload containing complete issue details.

The 10 Directory Validation Rules (L001–L010)

The validator implements each check as a dedicated function, assigning a unique rule code to every violation type.

L001: Directory Naming Convention

The check_lesson_dir_pattern function (line 85) validates that lesson folders follow the strict NN-slug pattern. This requires a two-digit phase number, followed by a hyphen, then a lowercase alphanumeric string that may contain hyphens but must end with an alphanumeric character.

L002–L004: Documentation Standards

The check_docs_en_md function (lines 97, 107, 114) verifies three critical aspects of the English documentation file:

• L002: Confirms the presence of docs/en.md and validates UTF-8 encoding.
• L003: Ensures the file exceeds 200 bytes to prevent placeholder documentation.
• L004: Validates that the markdown contains a top-level heading (# ...).

L005: Code Directory Requirements

The check_code_main function (line 119) inspects the code/ subdirectory to ensure it is not empty. The check ignores generic filenames and requires at least one actual source or configuration file to be present.

L006–L009: Quiz Schema Validation

The check_quiz function (lines 129, 157, 176, 186) performs comprehensive validation of quiz.json files:

• L006: Verifies valid JSON structure with a non-empty questions array (or a root-level list), ensuring each question object contains all required keys.
• L007: Detects usage of the deprecated legacy schema using fields q, choices, and answer.
• L008: Validates that each question's options array contains between 2 and 6 entries.
• L009: Confirms the correct field is an integer index pointing to a valid position within the options list.

L010: Internal Link Integrity

The check_internal_links function (line 196) parses docs/en.md to verify that all internal markdown links resolve to existing files or directories within the repository, preventing broken references in the curriculum.

Running the Audit Script

Execute the validator from the repository root to check all lessons:

python scripts/audit_lessons.py

To validate only a specific phase:

python scripts/audit_lessons.py --phase 02

For machine-readable output suitable for automated reporting:

python scripts/audit_lessons.py --json > audit_report.json

The script produces output similar to the following when issues are detected:

audit_lessons.py — 125 lesson(s) checked, 7 issue(s)

  [L002] phases/02-ml-basics/01-linear-regression/docs/en.md: missing docs/en.md
  [L005] phases/03-nn-basics/05-activation-functions/code: code/ is empty (no source or config files)
  [L009] phases/04-optimizers/02-sgd/quiz.json: question[3] correct=5 not a valid index in options[0..3]

Summary by rule:
  L002: 1
  L005: 1
  L009: 1

Summary

The audit_lessons.py script functions as the curriculum's structural enforcer, implementing ten specific validation rules:

• L001 enforces the NN-slug directory naming convention.
• L002–L004 ensure docs/en.md exists, is sufficiently large, and contains a proper heading.
• L005 verifies the code/ directory contains actual source files.
• L006–L009 validate quiz.json structure, schema compliance, option counts, and correct answer indices.
• L010 guarantees all internal documentation links resolve correctly.

Frequently Asked Questions

What is the primary purpose of audit_lessons.py?

The script serves as a lint-like validator that prevents structural errors, missing documentation, and malformed quizzes from entering the published curriculum. It automates quality control by verifying that every lesson directory under phases/ adheres to the repository's strict formatting and content standards.

How does the script validate quiz.json files?

The check_quiz function validates JSON syntax, ensures the presence of a questions array, checks for deprecated legacy schema usage, verifies that each question contains 2–6 options, and confirms that the correct index points to a valid option. These checks prevent broken quiz functionality in the learning platform.

Can I run checks on a specific subset of lessons?

Yes. The iter_lesson_dirs() function supports the --phase command-line argument, allowing you to filter validation to a specific phase directory. For example, python scripts/audit_lessons.py --phase 02 validates only the lessons within phases/02-ml-basics/.

Where are the validation rules implemented in the source code?

Each rule is implemented as a specific function in scripts/audit_lessons.py. Directory naming checks reside in check_lesson_dir_pattern (line 85), documentation checks in check_docs_en_md (lines 97–114), code requirements in check_code_main (line 119), quiz validation in check_quiz (lines 129–186), and link verification in check_internal_links (line 196).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}