Which Validation Checks Does the audit_lessons.py Script Perform?

The audit_lessons.py script performs 10 rule-based validation checks (L001–L010) that enforce consistent directory naming, documentation standards, code presence, quiz schema integrity, and internal link resolution across the AI Engineering from Scratch curriculum.

The audit_lessons.py script serves as the central lint-style validator for the rohitg00/ai-engineering-from-scratch repository. It traverses every lesson directory to ensure curriculum content meets strict structural and content standards before deployment. These validation checks guarantee that each lesson contains readable documentation, executable code, valid quizzes, and working internal hyperlinks.

The 10 Specific Validation Checks Performed by audit_lessons.py

The validator employs a rule-based system where each check emits a specific identifier (L001 through L010) when violations are detected. These rules are implemented as discrete helper functions within scripts/audit_lessons.py.

L001: Directory Naming Pattern

Rule L001 validates that every lesson directory conforms to the NN-slug naming convention. The check_lesson_dir_pattern function (lines 85–94) enforces a regular expression pattern of ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$, requiring a two-digit number prefix followed by a kebab-case slug.

L002–L004: Documentation Standards

The check_docs_en_md function (lines 97–115) performs three distinct validations on the docs/en.md file:

• L002: Verifies the file exists and can be read as UTF-8 text (lines 97–106)
• L003: Ensures the documentation meets the minimum size requirement of 200 bytes as defined by the MIN_DOC_BYTES constant (lines 107–113)
• L004: Confirms the presence of a top-level H1 heading using the markdown # syntax (lines 114–115)

L005: Code Directory Presence

Rule L005 ensures the code/ directory contains actual source material. The check_code_main function (lines 119–126) verifies the folder is not empty and contains at least one file that is not in the ignore list (which excludes README.md, .gitkeep, and similar non-code files).

L006–L009: Quiz Schema Validation

The check_quiz function (lines 129–193) performs four comprehensive checks on quiz.json:

• L006 (lines 129–152): Validates that the file is parseable JSON, contains a non-empty questions array, and that each question object includes the required canonical keys
• L007 (lines 156–166): Detects legacy schema keys (q, choices, answer) that have been deprecated in favor of the current standard
• L008 (lines 176–184): Validates that each question's options array contains between 2 and 6 items, as defined by the MIN_OPTIONS and MAX_OPTIONS constants
• L009 (lines 186–193): Verifies that the correct field contains an integer representing a valid index within the options list

L010: Internal Link Resolution

Rule L010 validates that all relative markdown links in docs/en.md resolve to existing files within the repository. The check_internal_links function (lines 96–112) ensures no broken internal hyperlinks reach production that would disrupt the generated site. For finer granularity link validation, the repository also includes the companion scripts/link_check.py utility.

How to Run the Validator Locally

Execute the full curriculum audit from the repository root:

python3 scripts/audit_lessons.py

To limit validation to a specific phase (for example, Phase 14) and output machine-readable JSON for CI integration:

python3 scripts/audit_lessons.py --phase 14 --json

CI Integration and Exit Codes

The script returns exit code 0 when all validation checks pass, and 1 when any rule violation is detected. This behavior allows GitHub Actions and other CI systems to automatically reject pull requests that introduce structural errors or content violations. The AGENTS.md file documents the audit workflow that invokes this validator, while README.md explains the "CI will reject the PR" enforcement policy.

Summary

• L001 enforces the NN-slug directory naming pattern via check_lesson_dir_pattern
• L002–L004 ensure docs/en.md exists, exceeds 200 bytes, and contains an H1 heading
• L005 confirms the code/ directory contains actual source files
• L006–L009 validate quiz.json structure, schema compliance, option counts (2–6), and correct answer indices
• L010 verifies all internal markdown links resolve to existing files
• The script exits with code 0 on success and 1 on failure, enabling automated CI gating

Frequently Asked Questions

What is the minimum documentation size required by the L003 check?

The L003 validation requires that docs/en.md contain at least 200 bytes of content. This minimum is defined by the MIN_DOC_BYTES constant in scripts/audit_lessons.py (lines 107–113) to ensure lessons contain substantive explanations rather than placeholder text.

Which files does the L005 code presence check ignore?

The L005 check ignores common non-code files such as README.md, .gitkeep, and other metadata files when determining if the code/ directory is empty. The check_code_main function (lines 119–126) specifically filters these out to ensure only actual source or configuration files satisfy the requirement.

How does the script handle invalid quiz JSON structure?

If L006 detects that quiz.json is malformed JSON, missing the questions array, or lacking required canonical keys, the validator emits an error and halts processing of that specific quiz file. The check_quiz function (lines 129–152) validates JSON parseability before checking schema compliance, ensuring the CI receives clear feedback about file corruption versus content violations.

Can I run the audit on a specific phase only?

Yes. Use the --phase flag followed by the phase number to restrict validation to a single phase. For example, python3 scripts/audit_lessons.py --phase 14 validates only Phase 14 lessons. Combine with --json to output results in a machine-readable format suitable for CI pipelines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}