# Which Validation Checks Does the audit_lessons.py Script Perform?

> Discover 10 rule-based validation checks (L001–L010) performed by audit_lessons.py. Ensure consistent naming, documentation, code, quizzes, and links in AI Engineering from Scratch.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-18

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script performs 10 rule-based validation checks (L001–L010) that enforce consistent directory naming, documentation standards, code presence, quiz schema integrity, and internal link resolution across the AI Engineering from Scratch curriculum.**

The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script serves as the central lint-style validator for the **rohitg00/ai-engineering-from-scratch** repository. It traverses every lesson directory to ensure curriculum content meets strict structural and content standards before deployment. These validation checks guarantee that each lesson contains readable documentation, executable code, valid quizzes, and working internal hyperlinks.

## The 10 Specific Validation Checks Performed by audit_lessons.py

The validator employs a rule-based system where each check emits a specific identifier (L001 through L010) when violations are detected. These rules are implemented as discrete helper functions within [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py).

### L001: Directory Naming Pattern

**Rule L001** validates that every lesson directory conforms to the `NN-slug` naming convention. The `check_lesson_dir_pattern` function (lines 85–94) enforces a regular expression pattern of `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`, requiring a two-digit number prefix followed by a kebab-case slug.

### L002–L004: Documentation Standards

The `check_docs_en_md` function (lines 97–115) performs three distinct validations on the [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file:

- **L002**: Verifies the file exists and can be read as **UTF-8** text (lines 97–106)
- **L003**: Ensures the documentation meets the minimum size requirement of **200 bytes** as defined by the `MIN_DOC_BYTES` constant (lines 107–113)
- **L004**: Confirms the presence of a top-level **H1 heading** using the markdown `#` syntax (lines 114–115)

### L005: Code Directory Presence

**Rule L005** ensures the `code/` directory contains actual source material. The `check_code_main` function (lines 119–126) verifies the folder is not empty and contains at least one file that is not in the ignore list (which excludes [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), `.gitkeep`, and similar non-code files).

### L006–L009: Quiz Schema Validation

The `check_quiz` function (lines 129–193) performs four comprehensive checks on [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json):

- **L006** (lines 129–152): Validates that the file is parseable JSON, contains a non-empty `questions` array, and that each question object includes the required canonical keys
- **L007** (lines 156–166): Detects legacy schema keys (`q`, `choices`, `answer`) that have been deprecated in favor of the current standard
- **L008** (lines 176–184): Validates that each question's `options` array contains between **2 and 6** items, as defined by the `MIN_OPTIONS` and `MAX_OPTIONS` constants
- **L009** (lines 186–193): Verifies that the `correct` field contains an integer representing a valid index within the `options` list

### L010: Internal Link Resolution

**Rule L010** validates that all relative markdown links in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) resolve to existing files within the repository. The `check_internal_links` function (lines 96–112) ensures no broken internal hyperlinks reach production that would disrupt the generated site. For finer granularity link validation, the repository also includes the companion [`scripts/link_check.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/link_check.py) utility.

## How to Run the Validator Locally

Execute the full curriculum audit from the repository root:

```bash
python3 scripts/audit_lessons.py

```

To limit validation to a specific phase (for example, Phase 14) and output machine-readable JSON for CI integration:

```bash
python3 scripts/audit_lessons.py --phase 14 --json

```

## CI Integration and Exit Codes

The script returns exit code **0** when all validation checks pass, and **1** when any rule violation is detected. This behavior allows GitHub Actions and other CI systems to automatically reject pull requests that introduce structural errors or content violations. The [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) file documents the `audit` workflow that invokes this validator, while [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) explains the "CI will reject the PR" enforcement policy.

## Summary

- **L001** enforces the `NN-slug` directory naming pattern via `check_lesson_dir_pattern`
- **L002–L004** ensure [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists, exceeds 200 bytes, and contains an H1 heading
- **L005** confirms the `code/` directory contains actual source files
- **L006–L009** validate [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) structure, schema compliance, option counts (2–6), and correct answer indices
- **L010** verifies all internal markdown links resolve to existing files
- The script exits with code **0** on success and **1** on failure, enabling automated CI gating

## Frequently Asked Questions

### What is the minimum documentation size required by the L003 check?

The **L003** validation requires that [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) contain at least **200 bytes** of content. This minimum is defined by the `MIN_DOC_BYTES` constant in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) (lines 107–113) to ensure lessons contain substantive explanations rather than placeholder text.

### Which files does the L005 code presence check ignore?

The **L005** check ignores common non-code files such as [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), `.gitkeep`, and other metadata files when determining if the `code/` directory is empty. The `check_code_main` function (lines 119–126) specifically filters these out to ensure only actual source or configuration files satisfy the requirement.

### How does the script handle invalid quiz JSON structure?

If **L006** detects that [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) is malformed JSON, missing the `questions` array, or lacking required canonical keys, the validator emits an error and halts processing of that specific quiz file. The `check_quiz` function (lines 129–152) validates JSON parseability before checking schema compliance, ensuring the CI receives clear feedback about file corruption versus content violations.

### Can I run the audit on a specific phase only?

Yes. Use the `--phase` flag followed by the phase number to restrict validation to a single phase. For example, `python3 scripts/audit_lessons.py --phase 14` validates only Phase 14 lessons. Combine with `--json` to output results in a machine-readable format suitable for CI pipelines.