How the Audit Script in rohitg00/ai-engineering-from-scratch Validates Lesson Structure Invariants

The audit_lessons.py script in the rohitg00/ai-engineering-from-scratch repository enforces structural invariants across 500+ lessons by running a deterministic eight-step pipeline that checks directory naming, documentation requirements, code presence, quiz schemas, and internal links, aggregating violations into rule-coded issues ranging from L001 to L010.

The rohitg00/ai-engineering-from-scratch curriculum organizes over 500 lessons across multiple phases, requiring strict structural consistency to remain maintainable. The audit_lessons.py script serves as the central lint-like validator that ensures every lesson directory adheres to naming conventions, contains required documentation with valid markdown, and follows quiz schema standards. By validating these lesson structure invariants automatically, the script guarantees that content contributors cannot accidentally merge malformed lessons into the main branch.

Lesson Discovery and Naming Conventions

The validation process begins with directory discovery. Starting at the repository ROOT (defined at lines 22–27), the script traverses the phases/ folder using iter_lesson_dirs() (lines 65–83) to identify directories matching the strict regex pattern ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$.

Each phase and lesson must follow the NN-slug naming convention. The check_lesson_dir_pattern() function (lines 85–94) validates this format, emitting issue L001 and skipping any malformed directories to prevent cascading validation errors.

Documentation Validation (docs/en.md)

Every lesson must contain a docs/en.md file meeting four strict criteria enforced by check_docs_en_md() (lines 97–116):

• Presence: The file must exist (L002)
• Encoding: Must be valid UTF-8 (L002)
• Minimum size: At least 200 bytes (L003)
• Top-level heading: Must contain a markdown H1 (# …) (L004)

The function returns the file text for subsequent link validation, ensuring downstream checks operate on verified content.

Code and Quiz Schema Enforcement

Non-empty Code Directories

If a code/ directory exists, it must contain at least one substantive file. The check_code_main() function (lines 119–127) excludes ignored files like README.md and .gitkeep, raising issue L005 if the directory is empty or contains only placeholder files.

Quiz.json Schema Validation

The quiz.json validator enforces a strict schema through check_quiz() (lines 129–194). The script parses the JSON and validates that the structure is either a list of questions or an object containing a questions array.

Each question must satisfy:

• Canonical keys only: Using stage, question, options, correct, explanation—legacy keys trigger L007
• Required fields: Missing keys generate L006
• Options length: Between 2 and 6 items (L008)
• Valid correct index: Must be an integer within the options array bounds (L009)

Internal Link Resolution

After successfully reading docs/en.md, the script verifies all markdown links using check_internal_links() (lines 196–212). The resolver ignores absolute URLs, mailto, and data URIs, while resolving relative paths against the document's parent directory or repository root (for / prefixed links). Missing targets generate issue L010, which is shared with the companion scripts/link_check.py utility for dedicated intra-repo link validation.

Issue Aggregation and Reporting

The Audit class (lines 38–63) maintains a collection of Issue objects tracking violations across the curriculum. The add() method normalizes all paths to repository-relative POSIX format, ensuring consistent reporting across operating systems.

The render_report() function (lines 25–42) outputs results either as JSON (with --json flag) or human-readable text. The main() entry point (lines 44–73) orchestrates the validation flow and exits with status code 1 when violations exist.

Run the audit from the repository root:

python scripts/audit_lessons.py        # Plain text report

python scripts/audit_lessons.py --json # JSON output

python scripts/audit_lessons.py --phase 5  # Validate only Phase 5

Sample output shows rule-coded violations:

audit_lessons.py — 87 lesson(s) checked, 4 issue(s)

  [L004] phases/09-reinforcement-learning/01-q-learning/docs/en.md: docs/en.md missing top-level H1
  [L008] phases/12-transformers/03-multihead-attention/quiz.json: question[2] options length must be 2..6 (got 1)

Summary by rule:
  L004: 1
  L008: 1

Summary

• The audit_lessons.py script in rohitg00/ai-engineering-from-scratch validates lesson structure invariants through an eight-step deterministic pipeline defined in scripts/audit_lessons.py.
• Directory naming is enforced via regex patterns in iter_lesson_dirs() (lines 65–83) and check_lesson_dir_pattern() (lines 85–94), with violations marked as L001.
• Documentation standards require UTF-8 encoded docs/en.md files exceeding 200 bytes with proper H1 headings (L002–L004).
• Code directories must contain substantive files (L005), while quiz.json must follow strict schema rules for questions, options, and correct indices (L006–L009).
• Internal links are resolved and validated to prevent broken references (L010).
• The script aggregates all violations into an Audit object and exits with status 1 when structural invariants are violated, preventing regression in the 500+ lesson curriculum.

Frequently Asked Questions

What triggers an L001 error in the audit script?

An L001 error occurs when a lesson directory name fails to match the required NN-slug pattern (^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$). The check_lesson_dir_pattern() function detects this mismatch and skips further validation for that directory to prevent invalid path processing.

How does the audit script validate quiz.json files?

The check_quiz() function (lines 129–194) parses the JSON and validates that each question uses the canonical schema keys (stage, question, options, correct, explanation), contains all required fields, provides 2–6 options, and specifies a valid correct index. Violations are reported as L006 through L009.

Can the audit script check only specific phases?

Yes. Running python scripts/audit_lessons.py --phase 5 limits validation to Phase 5 only. The script filters lessons during the discovery phase in iter_lesson_dirs() before applying any invariant checks, optimizing validation time for large repositories.

What is the difference between issues L002 and L004?

L002 covers file existence and UTF-8 encoding failures in docs/en.md, while L004 specifically flags missing top-level H1 headings within that file. A lesson could pass encoding checks but still fail if it lacks the required markdown header structure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}