What Does audit_lessons.py Validate in the AI Engineering Curriculum Structure?

The audit_lessons.py script enforces structural invariants across the AI Engineering from Scratch curriculum, validating directory naming, documentation standards, code presence, quiz schemas, and internal link resolution.

The audit_lessons.py script is the primary invariant-checking utility for the AI Engineering from Scratch repository maintained by rohitg00. Located at scripts/audit_lessons.py, it systematically traverses every lesson directory under phases/ to ensure the curriculum remains consistent, discoverable, and renderable by the static site builder.

Lesson Directory Naming Convention (L001)

The script validates that every lesson directory matches the strict pattern NN-slug using the regex ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$. This guarantees deterministic ordering and predictable URL construction for the generated site. Any directory failing this pattern is flagged as a violation of rule L001.

Documentation Integrity Standards (L002–L004, L010)

The audit enforces strict requirements for lesson documentation files:

• L002–L004: The file docs/en.md must exist, be encoded in UTF-8, contain at least 200 bytes of content, and include a top-level H1 heading (# …). These rules ensure the human-readable description is present and correctly formatted for both learners and the static-site generator.

• L010: All internal Markdown links within docs/en.md must resolve to existing files (relative or absolute). This prevents broken navigation links that would degrade the learning experience.

Code Artifact Requirements (L005)

Every lesson must ship with runnable example code. The script checks that the code/ directory contains at least one source file, excluding ignored names. An empty code/ folder triggers violation L005, which would break the repository’s "Run → Test" workflow.

Quiz Schema Validation (L006–L009)

The quiz.json file undergoes rigorous schema checking to ensure compatibility with the quiz UI:

• L006: The file must be valid JSON containing a non-empty questions array.
• L007: The script detects legacy quiz keys (q, choices, answer) that require migration to the current schema.
• L008: Each question must have between 2 and 6 options to prevent UI overflow or under-populated questions.
• L009: The correct index must be a valid integer within the bounds of the options list.

The canonical schema requires fields: stage, question, options, correct, and explanation.

Running the Audit Locally

Execute the validator from the repository root to check the entire curriculum or specific phases:

# Check every lesson

python scripts/audit_lessons.py

# Restrict to Phase 5 only

python scripts/audit_lessons.py --phase 5

# Output JSON for automated processing

python scripts/audit_lessons.py --json > report.json

The script aggregates results into a human-readable report (or JSON with --json) and exits with a non-zero status if any violations are found, making it suitable for CI enforcement.

Sample output format:

audit_lessons.py — 435 lesson(s) checked, 3 issue(s)

  [L005] phases/03-foundations/09-linear-regression/code: code/ is empty
  [L010] phases/07-transformers/13-attention-mechanisms/docs/en.md: internal link does not resolve

CI/CD Pipeline Integration

According to the source code in .github/workflows/curriculum.yml, the audit runs automatically in the audit workflow to gate contributions. The script works alongside scripts/check_readme_counts.py (which validates README lesson counts) and site/build.js (which generates site/data.js and relies on the validated links).

Summary

• Directory Structure: Enforces NN-slug naming convention (L001) for deterministic ordering.
• Documentation: Requires docs/en.md with H1, UTF-8 encoding, minimum size, and valid internal links (L002–L004, L010).
• Code Presence: Mandates at least one source file in the code/ directory (L005).
• Quiz Integrity: Validates quiz.json schema, legacy key detection, and option bounds (L006–L009).
• Automation: Supports JSON output and phase-specific filtering for CI/CD integration.

Frequently Asked Questions

Where is the audit_lessons.py script located in the repository?

The script is located at scripts/audit_lessons.py in the root of the rohitg00/ai-engineering-from-scratch repository. It serves as the single source of truth for lesson structure validation.

What specific schema does the script enforce for quiz.json files?

The script requires quiz.json to contain a valid questions array where each object includes stage, question, options, correct, and explanation fields. Options must number between 2 and 6, and the correct value must be a valid index into the options array.

How does the script validate internal documentation links?

Rule L010 parses all Markdown links in docs/en.md and verifies they resolve to existing files on the filesystem, checking both relative and absolute paths to prevent broken navigation.

Can the audit be restricted to specific curriculum phases?

Yes. Use the --phase flag followed by the phase number to validate only lessons within that phase (e.g., python scripts/audit_lessons.py --phase 5), which is useful for incremental checks in large repositories.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}