What Does `scripts/audit_lessons.py` Validate? Curriculum Compliance Rules Explained

The scripts/audit_lessons.py script validates that every lesson in the rohitg00/ai-engineering-from-scratch repository conforms to ten strict structural, content, and schema compliance rules (L001-L010), ensuring directory naming consistency, documentation quality, quiz integrity, and internal link resolution before code reaches the CI pipeline.

Maintaining consistency across hundreds of lessons in an open-source curriculum requires automated enforcement. The scripts/audit_lessons.py utility acts as a lint-style validator that audits the entire rohitg00/ai-engineering-from-scratch repository against canonical formatting standards. It guarantees that every lesson directory contains properly structured documentation, functional code examples, valid assessment files, and resolvable internal references.

Directory and File Structure Validation (L001-L005)

The validator first ensures that lesson containers and mandatory files follow the repository's organizational schema.

Lesson Naming Conventions (L001)

Every lesson directory must match the NN-slug pattern (e.g., 00-introduction, 12-transformers). In scripts/audit_lessons.py, the LESSON_DIR_RE regex pattern (lines 25-31) defines the allowed format, while the check_lesson_dir_pattern() function (lines 85-94) executes this validation against each lesson path. Directories that fail to use the two-digit prefix and kebab-case suffix trigger an immediate compliance violation.

Documentation Standards (L002-L004)

Each lesson must include a docs/en.md file meeting three specific criteria enforced by the check_docs_en_md() function:

• L002: The file must exist and be valid UTF-8. The validator attempts to decode the file and catches UnicodeDecodeError to detect encoding issues (lines 98-106).
• L003: Documentation must be substantive—minimum 200 bytes. This check at lines 107-113 prevents placeholder or empty content from entering the curriculum.
• L004: The markdown must contain a top-level H1 heading (# ...). A regex test at lines 114-115 verifies this requirement, ensuring consistent document hierarchy for rendering.

Code Directory Integrity (L005)

Lessons cannot submit empty code/ directories. The check_code_main() function (lines 124-126) scans each lesson's code folder to confirm it contains at least one non-ignored source or configuration file (with validation logic defined at lines 19-26). This prevents incomplete lessons from passing review.

Quiz Schema and Content Validation (L006-L009)

The validator enforces strict data integrity rules on assessment files using the check_quiz() function.

JSON Structure and Legacy Detection (L006-L007)

• L006: The quiz.json file must parse as valid JSON and contain a non-empty questions array. Each question must include all required canonical keys: stage, question, options, correct, and explanation (lines 30-54, 56-71).
• L007: The script detects legacy quiz schemas that use deprecated keys (q, choices, answer) and flags them for migration to the current format (lines 57-65).

Answer Key Integrity (L008-L009)

The validator ensures that quizzes are logically consistent and answerable:

• L008: Each question's options array must contain between 2 and 6 entries (lines 76-84).
• L009: The correct field must be an integer index that points to a valid position within the options array (lines 86-93).

Internal Link Validation (L010)

Broken cross-references disrupt the learning experience. The check_internal_links() function (lines 96-112) scans all markdown files for internal references using the pattern [text](href). It resolves both relative and absolute paths to verify that every link targets an existing file or directory within the repository. This check can also be run independently using the companion scripts/link_check.py utility.

Running the Curriculum Audit

Execute the validator from the repository root to scan all lesson directories:

python scripts/audit_lessons.py

The script aggregates all findings into an Audit object and returns exit code 0 for a clean repository or 1 when violations exist, enabling CI pipelines to fail fast.

Generate a machine-readable JSON report for programmatic processing:

python scripts/audit_lessons.py --json > audit_report.json

The JSON payload includes lessons_checked and an array of issue objects with rule, lesson, file, and message fields.

Limit validation to a specific curriculum phase:

python scripts/audit_lessons.py --phase 5

Enable strict mode to treat all warnings as errors:

python scripts/audit_lessons.py --strict

Summary

• scripts/audit_lessons.py enforces ten compliance rules (L001-L010) across the entire rohitg00/ai-engineering-from-scratch curriculum.
• Structural checks validate the NN-slug directory naming pattern, mandatory docs/en.md presence, 200-byte minimum documentation size, required H1 headings, and non-empty code/ folders.
• Quiz validation ensures JSON parses correctly, uses current schema keys (not legacy q/choices), provides 2-6 options per question, and indexes correct answers within valid bounds.
• Link integrity verification confirms all internal markdown references resolve to existing repository paths via check_internal_links().
• The script supports CI integration through exit codes (0 or 1) and provides JSON output for automated reporting pipelines.

Frequently Asked Questions

What happens if a lesson directory fails the L001 naming check?

The audit reports a structural violation when a directory does not match the NN-slug pattern defined by LESSON_DIR_RE in scripts/audit_lessons.py. You must rename the folder to use a two-digit numeric prefix followed by a kebab-case descriptor (e.g., 03-nlp-basics) before the CI pipeline will pass validation.

Can I validate quiz files separately from the full audit?

While scripts/audit_lessons.py runs all ten checks together, the quiz-specific logic is encapsulated in the check_quiz() function (lines 30-93). For isolated link validation (rule L010), use the companion scripts/link_check.py script which implements the link resolution checks independently.

Why does the validator require documentation to be at least 200 bytes?

Rule L003 enforces substantive lesson descriptions by checking file size in check_docs_en_md() (lines 107-113). This minimum threshold prevents placeholder or trivial documentation files from entering the curriculum, ensuring every lesson provides meaningful explanatory content for learners.

How does the script detect obsolete quiz formats?

Rule L007 scans for legacy schema keys (q, choices, answer) during quiz.json parsing (lines 57-65). When detected, the audit flags these structures so maintainers can migrate them to the current canonical format using stage, question, options, correct, and explanation keys.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}