Most Common Validation Issues Caught by audit_lessons.py in AI-Engineering-From-Scratch

The audit_lessons.py script detects ten structural violations (L001–L010) in the rohitg00/ai-engineering-from-scratch curriculum, with naming errors, missing documentation, and malformed quizzes accounting for the majority of failures.

The audit_lessons.py validator serves as the canonical invariant checker for every lesson directory in the rohitg00/ai-engineering-from-scratch repository. When executed, it scans the entire curriculum—or a single phase—to surface specific validation issues caught by the audit_lessons.py script before they reach production. Understanding these rules helps contributors fix scaffolding errors ranging from invalid folder names to broken internal links.

How the Audit Script Works

Located at scripts/audit_lessons.py, the validator recursively inspects all lesson directories under the phases/ folder. It applies ten numbered rules (L001 through L010) that enforce naming conventions, documentation standards, code presence, and quiz schema validity. When run against the full curriculum, the script checks hundreds of lessons—reporting violations with specific file paths and error codes that map directly to the rule definitions.

The Five Most Common Validation Issues

While audit_lessons.py implements ten distinct checks, five rules surface most frequently in practice because they guard the fundamental scaffolding of every lesson.

L001: Invalid Lesson Directory Names

Rule L001 enforces that every lesson folder matches the strict NN-slug pattern: ^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$. This requires a two-digit number prefix, a dash, and a lowercase alphanumeric slug without underscores. Failures typically occur when contributors omit the leading zero (e.g., 7-lesson instead of 07-lesson) or use snake_case instead of kebab-case.

L002: Missing Documentation Files

Rule L002 verifies the existence of docs/en.md within each lesson directory. This file serves as the primary English documentation, and its absence triggers an immediate failure. The check ensures that no lesson enters the curriculum without a dedicated markdown guide.

L004: Missing Top-Level H1 Headers

Rule L004 inspects docs/en.md for a visible title by requiring a top-level H1 heading (# Title). Authors often forget to add this header after creating the file, resulting in documentation that lacks a proper title node for rendering.

L005: Empty Code Directories

Rule L005 confirms that the code/ directory contains at least one source file, excluding ignored placeholder names like .gitkeep. Empty folders or directories containing only temporary files violate this rule, ensuring every lesson includes executable reference material.

L006: Malformed Quiz JSON Structure

Rule L006 validates that quiz.json contains well-formed JSON and includes the required canonical keys: questions, question, options, correct, and explanation. Malformed JSON syntax or missing fields trigger this violation, preventing broken assessment data from propagating.

Additional Validation Rules (L003, L007–L010)

The remaining rules complete the integrity check:

• L003: Enforces a minimum size of 200 bytes for docs/en.md to prevent skeleton documentation containing only front-matter.
• L007: Prohibits legacy quiz keys (q, choices, answer) that were deprecated in favor of the current schema.
• L008: Validates that options arrays contain between 2 and 6 answer choices.
• L009: Confirms that the correct field is an integer index pointing to a valid position within the options array.
• L010: Resolves all internal Markdown links to verify they point to existing files, catching broken relative paths.

Running the Validator

Execute the audit from the repository root to check the entire curriculum:

python3 scripts/audit_lessons.py

Typical output includes the lesson count, issue tally, and specific failures:

audit_lessons.py — 435 lesson(s) checked, 27 issue(s)

  [L001] phases/07-computer-vision/7-incorrect-folder-name: lesson dir name does not match NN-slug pattern: '7-incorrect-folder-name'
  [L004] phases/04-computer-vision/04-sam3-open-vocab-segmentation/docs/en.md: docs/en.md missing top-level H1
  [L006] phases/02-ml-fundamentals/12-hyperparameter-tuning/quiz.json: question[2] missing keys ['correct', 'explanation', 'options', 'question', 'stage']
  [L010] phases/01-math-foundations/20-fourier-transform/docs/en.md: internal link does not resolve: '../03-monte-carlo-methods/'

For CI pipelines, output structured JSON:

python3 scripts/audit_lessons.py --json > audit-report.json

The JSON format emits an array of violation objects:

{
  "rule": "L004",
  "lesson": "phases/04-computer-vision/04-sam3-open-vocab-segmentation",
  "file": "phases/04-computer-vision/04-sam3-open-vocab-segmentation/docs/en.md",
  "message": "docs/en.md missing top-level H1"
}

Summary

• audit_lessons.py enforces ten structural rules (L001–L010) across all lesson directories in rohitg00/ai-engineering-from-scratch.
• The most frequent violations involve lesson naming (L001), missing documentation (L002), absent H1 headers (L004), empty code folders (L005), and malformed quizzes (L006).
• Run the script with python3 scripts/audit_lessons.py to receive human-readable reports, or use --json for programmatic processing in CI/CD pipelines.
• Each error code maps to a specific file path and remediation step, enabling rapid correction of curriculum scaffolding issues.

Frequently Asked Questions

What does the L001 validation error mean in audit_lessons.py?

L001 indicates that a lesson directory name does not match the required NN-slug pattern. The folder must start with a two-digit number (including leading zeros), followed by a hyphen and a lowercase alphanumeric string using hyphens as separators. For example, 07-computer-vision passes while 7-computer_vision fails.

How can I fix L006 quiz validation errors?

L006 errors occur when quiz.json contains invalid JSON syntax or lacks required keys. Ensure the file contains a top-level questions array where each object includes question, options, correct, and explanation fields. Remove any legacy keys like q or choices to avoid simultaneous L007 violations.

Can I run audit_lessons.py on a single phase instead of the entire curriculum?

Yes. While the default behavior scans all lessons under phases/, the script supports targeting a specific phase directory to limit the scope. This is useful for incremental validation in large repositories where only one phase has been modified.

What is the minimum size requirement for documentation files?

Rule L003 requires that every docs/en.md file exceed 200 bytes. This prevents skeleton templates containing only YAML front-matter or placeholder text from passing validation. Add substantive content, code examples, or explanatory paragraphs to satisfy this requirement.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}