# What Validation Checks Does audit_lessons.py Perform?

> Audit_lessons.py script runs ten validation checks L001-L010 on AI Engineering curriculum lesson directories enforcing naming, documentation, code, quiz, and link integrity.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-21

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script performs ten distinct validation checks (L001–L010) on every lesson directory in the AI Engineering from Scratch curriculum, enforcing strict rules for directory naming, documentation standards, code presence, quiz schema integrity, and internal link validity.**

The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) file in the `rohitg00/ai-engineering-from-scratch` repository functions as the central "lint-style" validator for the curriculum. It recursively walks every lesson directory and applies rule-based checks that emit specific error codes when violations are found. Understanding what validation checks [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) performs is essential for contributors who need to ensure their lessons meet the automated quality gates before CI/CD approval.

## Directory Structure and Naming (L001)

### L001 – Lesson Directory Pattern

The validator ensures every lesson directory follows the strict `NN-slug` naming convention. In the `check_lesson_dir_pattern` function (lines 85–94), the script checks that directory names match the regular expression `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`. This guarantees consistent URL-friendly slugs across the curriculum, requiring a two-digit number prefix followed by a hyphen and lowercase alphanumeric characters.

## Documentation Requirements (L002–L004)

The `check_docs_en_md` function (lines 97–115) validates the presence and quality of the primary documentation file.

### L002 – UTF-8 Readable docs/en.md

The script verifies that [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists and can be opened and read as valid UTF-8 text. If the file is missing or encoding errors occur, the validator flags the lesson with error code L002.

### L003 – Minimum File Size

Each documentation file must contain substantial content. The constant `MIN_DOC_BYTES` is set to **200 bytes**, and the script rejects any [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) smaller than this threshold to prevent empty or placeholder documentation.

### L004 – Mandatory H1 Heading

Beyond mere presence, the file must contain a top-level Markdown heading. The validator scans for the `# ` pattern to ensure the lesson has a proper title for rendered documentation.

## Code Asset Verification (L005)

### L005 – Non-Empty Code Directory

The `check_code_main` function (lines 119–126) ensures the `code/` subdirectory contains actual learning materials. It verifies the folder is not empty by checking for at least one source or configuration file, explicitly ignoring common placeholders like [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and `.gitkeep`. This check guarantees that every lesson includes executable example code.

## Quiz Schema Validation (L006–L009)

The `check_quiz` function (lines 129–193) performs four distinct checks on [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) to ensure machine-readable assessment data.

### L006 – Valid JSON Structure

The validator parses [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) and confirms it contains a non-empty `questions` array (or dictionary with that key). It also verifies that each question object includes the required canonical keys defined by the current schema.

### L007 – Legacy Key Detection

The script rejects questions using deprecated schema keys. Specifically, it flags any question containing the legacy fields `q`, `choices`, or `answer`, enforcing migration to the current standardized format.

### L008 – Options Count Constraints

Each multiple-choice question must have between **2** and **6** options. The constants `MIN_OPTIONS` (2) and `MAX_OPTIONS` (6) define these bounds, ensuring questions are neither trivial nor overly complex.

### L009 – Correct Answer Index Validity

The validator checks that the `correct` field is an integer pointing to a valid zero-based index within the `options` array. This prevents out-of-bounds errors when the quiz is rendered by automated systems.

## Link Integrity (L010)

### L010 – Internal Link Resolution

The `check_internal_links` function validates that all relative Markdown links in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) resolve to existing files within the repository. This prevents broken internal hyperlinks that would otherwise corrupt the generated static site, ensuring all cross-references between lessons remain functional.

## Running the Validator

Execute the validator from the repository root to check the entire curriculum:

```bash
python3 scripts/audit_lessons.py

```

Limit validation to a specific phase (e.g., Phase 14) and output JSON for CI consumption:

```bash
python3 scripts/audit_lessons.py --phase 14 --json

```

The script returns exit code **0** when no issues are found, or **1** when any validation rule fails, enabling integration with GitHub Actions or other CI pipelines.

## Summary

- **L001** enforces the `NN-slug` directory naming pattern via regex validation.
- **L002–L004** ensure [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists, exceeds 200 bytes, and contains an H1 heading.
- **L005** verifies the `code/` directory contains actual source files, not just placeholders.
- **L006–L009** validate [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) structure, reject legacy keys, enforce 2–6 options per question, and confirm correct answer indices.
- **L010** guarantees all internal Markdown links resolve to existing repository files.
- The validator exits with code 1 on failure, making it suitable for automated CI rejection of non-compliant pull requests.

## Frequently Asked Questions

### What happens if my lesson directory fails the L001 naming check?

The pull request will fail CI validation because [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) requires all lesson directories to match the `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$` pattern. You must rename the directory to use a two-digit number prefix followed by a hyphen and lowercase alphanumeric characters before the CI will pass.

### How does the validator detect broken internal links (L010)?

The `check_internal_links` function parses all relative Markdown links in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) and verifies that the target file paths exist within the repository structure. If a link points to a non-existent file or uses an absolute URL when it should be relative, the validator emits error code L010.

### What is the minimum content requirement for documentation files?

According to the `MIN_DOC_BYTES` constant in [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py), every [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must be at least **200 bytes** in size (L003) and must contain a top-level H1 heading (L004). This ensures lessons have sufficient descriptive content for learners.

### Can I include multiple quiz files or variations in one lesson?

No, the validator specifically checks for a single [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file in the lesson root. The `check_quiz` function validates that this file contains a valid JSON structure with a `questions` array following the canonical schema, including 2–6 options per question and a valid `correct` index. Legacy key formats using `q`, `choices`, or `answer` are explicitly rejected under rule L007.