# Most Common Validation Issues Caught by audit_lessons.py in AI-Engineering-From-Scratch

> Discover common validation issues audited by audit_lessons.py in ai-engineering-from-scratch. Learn about naming errors, missing docs, and malformed quizzes.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-12

---

**The audit_lessons.py script detects ten structural violations (L001–L010) in the rohitg00/ai-engineering-from-scratch curriculum, with naming errors, missing documentation, and malformed quizzes accounting for the majority of failures.**

The **audit_lessons.py** validator serves as the canonical invariant checker for every lesson directory in the **rohitg00/ai-engineering-from-scratch** repository. When executed, it scans the entire curriculum—or a single phase—to surface specific validation issues caught by the audit_lessons.py script before they reach production. Understanding these rules helps contributors fix scaffolding errors ranging from invalid folder names to broken internal links.

## How the Audit Script Works

Located at [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), the validator recursively inspects all lesson directories under the `phases/` folder. It applies ten numbered rules (**L001** through **L010**) that enforce naming conventions, documentation standards, code presence, and quiz schema validity. When run against the full curriculum, the script checks hundreds of lessons—reporting violations with specific file paths and error codes that map directly to the rule definitions.

## The Five Most Common Validation Issues

While **audit_lessons.py** implements ten distinct checks, five rules surface most frequently in practice because they guard the fundamental scaffolding of every lesson.

### L001: Invalid Lesson Directory Names

Rule **L001** enforces that every lesson folder matches the strict **NN-slug** pattern: `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`. This requires a two-digit number prefix, a dash, and a lowercase alphanumeric slug without underscores. Failures typically occur when contributors omit the leading zero (e.g., `7-lesson` instead of `07-lesson`) or use snake_case instead of kebab-case.

### L002: Missing Documentation Files

Rule **L002** verifies the existence of [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) within each lesson directory. This file serves as the primary English documentation, and its absence triggers an immediate failure. The check ensures that no lesson enters the curriculum without a dedicated markdown guide.

### L004: Missing Top-Level H1 Headers

Rule **L004** inspects [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) for a visible title by requiring a top-level H1 heading (`# Title`). Authors often forget to add this header after creating the file, resulting in documentation that lacks a proper title node for rendering.

### L005: Empty Code Directories

Rule **L005** confirms that the `code/` directory contains at least one source file, excluding ignored placeholder names like `.gitkeep`. Empty folders or directories containing only temporary files violate this rule, ensuring every lesson includes executable reference material.

### L006: Malformed Quiz JSON Structure

Rule **L006** validates that [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) contains well-formed JSON and includes the required canonical keys: `questions`, `question`, `options`, `correct`, and `explanation`. Malformed JSON syntax or missing fields trigger this violation, preventing broken assessment data from propagating.

## Additional Validation Rules (L003, L007–L010)

The remaining rules complete the integrity check:

- **L003**: Enforces a minimum size of **200 bytes** for [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) to prevent skeleton documentation containing only front-matter.
- **L007**: Prohibits legacy quiz keys (`q`, `choices`, `answer`) that were deprecated in favor of the current schema.
- **L008**: Validates that `options` arrays contain between 2 and 6 answer choices.
- **L009**: Confirms that the `correct` field is an integer index pointing to a valid position within the `options` array.
- **L010**: Resolves all internal Markdown links to verify they point to existing files, catching broken relative paths.

## Running the Validator

Execute the audit from the repository root to check the entire curriculum:

```bash
python3 scripts/audit_lessons.py

```

Typical output includes the lesson count, issue tally, and specific failures:

```

audit_lessons.py — 435 lesson(s) checked, 27 issue(s)

  [L001] phases/07-computer-vision/7-incorrect-folder-name: lesson dir name does not match NN-slug pattern: '7-incorrect-folder-name'
  [L004] phases/04-computer-vision/04-sam3-open-vocab-segmentation/docs/en.md: docs/en.md missing top-level H1
  [L006] phases/02-ml-fundamentals/12-hyperparameter-tuning/quiz.json: question[2] missing keys ['correct', 'explanation', 'options', 'question', 'stage']
  [L010] phases/01-math-foundations/20-fourier-transform/docs/en.md: internal link does not resolve: '../03-monte-carlo-methods/'

```

For CI pipelines, output structured JSON:

```bash
python3 scripts/audit_lessons.py --json > audit-report.json

```

The JSON format emits an array of violation objects:

```json
{
  "rule": "L004",
  "lesson": "phases/04-computer-vision/04-sam3-open-vocab-segmentation",
  "file": "phases/04-computer-vision/04-sam3-open-vocab-segmentation/docs/en.md",
  "message": "docs/en.md missing top-level H1"
}

```

## Summary

- **audit_lessons.py** enforces ten structural rules (**L001–L010**) across all lesson directories in **rohitg00/ai-engineering-from-scratch**.
- The most frequent violations involve **lesson naming** (L001), **missing documentation** (L002), **absent H1 headers** (L004), **empty code folders** (L005), and **malformed quizzes** (L006).
- Run the script with `python3 scripts/audit_lessons.py` to receive human-readable reports, or use `--json` for programmatic processing in CI/CD pipelines.
- Each error code maps to a specific file path and remediation step, enabling rapid correction of curriculum scaffolding issues.

## Frequently Asked Questions

### What does the L001 validation error mean in audit_lessons.py?

**L001** indicates that a lesson directory name does not match the required **NN-slug** pattern. The folder must start with a two-digit number (including leading zeros), followed by a hyphen and a lowercase alphanumeric string using hyphens as separators. For example, `07-computer-vision` passes while `7-computer_vision` fails.

### How can I fix L006 quiz validation errors?

**L006** errors occur when [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) contains invalid JSON syntax or lacks required keys. Ensure the file contains a top-level `questions` array where each object includes `question`, `options`, `correct`, and `explanation` fields. Remove any legacy keys like `q` or `choices` to avoid simultaneous **L007** violations.

### Can I run audit_lessons.py on a single phase instead of the entire curriculum?

Yes. While the default behavior scans all lessons under `phases/`, the script supports targeting a specific phase directory to limit the scope. This is useful for incremental validation in large repositories where only one phase has been modified.

### What is the minimum size requirement for documentation files?

Rule **L003** requires that every [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file exceed **200 bytes**. This prevents skeleton templates containing only YAML front-matter or placeholder text from passing validation. Add substantive content, code examples, or explanatory paragraphs to satisfy this requirement.