What Validation Checks Does audit_lessons.py Perform?

The audit_lessons.py script performs ten distinct validation checks (L001–L010) on every lesson directory in the AI Engineering from Scratch curriculum, enforcing strict rules for directory naming, documentation standards, code presence, quiz schema integrity, and internal link validity.

The audit_lessons.py file in the rohitg00/ai-engineering-from-scratch repository functions as the central "lint-style" validator for the curriculum. It recursively walks every lesson directory and applies rule-based checks that emit specific error codes when violations are found. Understanding what validation checks audit_lessons.py performs is essential for contributors who need to ensure their lessons meet the automated quality gates before CI/CD approval.

Directory Structure and Naming (L001)

L001 – Lesson Directory Pattern

The validator ensures every lesson directory follows the strict NN-slug naming convention. In the check_lesson_dir_pattern function (lines 85–94), the script checks that directory names match the regular expression ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$. This guarantees consistent URL-friendly slugs across the curriculum, requiring a two-digit number prefix followed by a hyphen and lowercase alphanumeric characters.

Documentation Requirements (L002–L004)

The check_docs_en_md function (lines 97–115) validates the presence and quality of the primary documentation file.

L002 – UTF-8 Readable docs/en.md

The script verifies that docs/en.md exists and can be opened and read as valid UTF-8 text. If the file is missing or encoding errors occur, the validator flags the lesson with error code L002.

L003 – Minimum File Size

Each documentation file must contain substantial content. The constant MIN_DOC_BYTES is set to 200 bytes, and the script rejects any docs/en.md smaller than this threshold to prevent empty or placeholder documentation.

L004 – Mandatory H1 Heading

Beyond mere presence, the file must contain a top-level Markdown heading. The validator scans for the # pattern to ensure the lesson has a proper title for rendered documentation.

Code Asset Verification (L005)

L005 – Non-Empty Code Directory

The check_code_main function (lines 119–126) ensures the code/ subdirectory contains actual learning materials. It verifies the folder is not empty by checking for at least one source or configuration file, explicitly ignoring common placeholders like README.md and .gitkeep. This check guarantees that every lesson includes executable example code.

Quiz Schema Validation (L006–L009)

The check_quiz function (lines 129–193) performs four distinct checks on quiz.json to ensure machine-readable assessment data.

L006 – Valid JSON Structure

The validator parses quiz.json and confirms it contains a non-empty questions array (or dictionary with that key). It also verifies that each question object includes the required canonical keys defined by the current schema.

L007 – Legacy Key Detection

The script rejects questions using deprecated schema keys. Specifically, it flags any question containing the legacy fields q, choices, or answer, enforcing migration to the current standardized format.

L008 – Options Count Constraints

Each multiple-choice question must have between 2 and 6 options. The constants MIN_OPTIONS (2) and MAX_OPTIONS (6) define these bounds, ensuring questions are neither trivial nor overly complex.

L009 – Correct Answer Index Validity

The validator checks that the correct field is an integer pointing to a valid zero-based index within the options array. This prevents out-of-bounds errors when the quiz is rendered by automated systems.

Link Integrity (L010)

L010 – Internal Link Resolution

The check_internal_links function validates that all relative Markdown links in docs/en.md resolve to existing files within the repository. This prevents broken internal hyperlinks that would otherwise corrupt the generated static site, ensuring all cross-references between lessons remain functional.

Running the Validator

Execute the validator from the repository root to check the entire curriculum:

python3 scripts/audit_lessons.py

Limit validation to a specific phase (e.g., Phase 14) and output JSON for CI consumption:

python3 scripts/audit_lessons.py --phase 14 --json

The script returns exit code 0 when no issues are found, or 1 when any validation rule fails, enabling integration with GitHub Actions or other CI pipelines.

Summary

• L001 enforces the NN-slug directory naming pattern via regex validation.
• L002–L004 ensure docs/en.md exists, exceeds 200 bytes, and contains an H1 heading.
• L005 verifies the code/ directory contains actual source files, not just placeholders.
• L006–L009 validate quiz.json structure, reject legacy keys, enforce 2–6 options per question, and confirm correct answer indices.
• L010 guarantees all internal Markdown links resolve to existing repository files.
• The validator exits with code 1 on failure, making it suitable for automated CI rejection of non-compliant pull requests.

Frequently Asked Questions

What happens if my lesson directory fails the L001 naming check?

The pull request will fail CI validation because audit_lessons.py requires all lesson directories to match the ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$ pattern. You must rename the directory to use a two-digit number prefix followed by a hyphen and lowercase alphanumeric characters before the CI will pass.

How does the validator detect broken internal links (L010)?

The check_internal_links function parses all relative Markdown links in docs/en.md and verifies that the target file paths exist within the repository structure. If a link points to a non-existent file or uses an absolute URL when it should be relative, the validator emits error code L010.

What is the minimum content requirement for documentation files?

According to the MIN_DOC_BYTES constant in audit_lessons.py, every docs/en.md must be at least 200 bytes in size (L003) and must contain a top-level H1 heading (L004). This ensures lessons have sufficient descriptive content for learners.

Can I include multiple quiz files or variations in one lesson?

No, the validator specifically checks for a single quiz.json file in the lesson root. The check_quiz function validates that this file contains a valid JSON structure with a questions array following the canonical schema, including 2–6 options per question and a valid correct index. Legacy key formats using q, choices, or answer are explicitly rejected under rule L007.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}