# Validation Rules Enforced by audit_lessons.py on quiz.json and Lesson Frontmatter

> Discover the 10 validation rules audit_lessons.py enforces on quiz.json and lesson frontmatter for consistent curriculum in ai-engineering-from-scratch.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-08-29

---

**The [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) script enforces ten specific validation rules (L001-L010) that check lesson directory naming, documentation structure, and quiz JSON schema to ensure curriculum consistency across the ai-engineering-from-scratch repository.**

The `rohitg00/ai-engineering-from-scratch` repository maintains strict structural standards for its educational content through automated validation. The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) file implements a comprehensive audit suite that validates every lesson directory against invariant checks, specifically targeting **[`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json)** schema compliance and **[`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md)** frontmatter integrity. These rules ensure that all curriculum content remains machine-readable, internally consistent, and free from legacy formatting errors.

## Directory Structure and Naming Conventions

Before inspecting content, the auditor validates the physical organization of lesson directories according to strict naming conventions.

### Lesson Directory Naming (L001)

Every lesson folder must match the regex pattern `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`, enforcing the `NN-slug` format where `NN` is a two-digit number. This rule, implemented in lines **85‑94** of [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), ensures consistent sorting and URL-friendly identifiers across the `phases/` directory structure.

### Documentation Requirements (L002-L004)

The auditor performs a three-tier validation on [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md):

1. **Existence Check (L002)**: The file must exist at `phases/*/*/docs/en.md` (lines **99‑102**).
2. **Size Validation (L003)**: The document must contain at least `MIN_DOC_BYTES` (200 bytes), preventing empty or stub files (lines **103‑114**).
3. **Header Verification (L004)**: The Markdown must contain a top-level H1 heading (`# …`) to ensure proper document structure (lines **115‑116**).

### Code Directory Validation (L005)

The `code/` subdirectory cannot be empty; it must contain at least one non-ignored source file. This check, found in lines **119‑126**, ensures that every lesson includes practical implementation materials alongside theoretical documentation.

## Quiz JSON Schema Validation

The most rigorous checks target [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files, enforcing both syntactic correctness and semantic schema compliance.

### Structure and Syntax (L006)

Lines **129‑152** validate that the JSON is syntactically valid and follows one of two allowed top-level structures:

- A non-empty list of question objects, or
- An object containing a non-empty `questions` array

Additionally, lines **167‑174** verify that each question contains the required canonical keys: `stage`, `question`, `options`, `correct`, and `explanation`.

### Legacy Schema Detection (L007)

The auditor actively prohibits deprecated keys to prevent schema drift. Lines **157‑166** disallow the legacy keys `q`, `choices`, and `answer`, forcing migration to the current canonical schema. If detected, the audit fails with a specific error identifying the obsolete field.

### Options Field Constraints (L008)

The `options` array must contain between `MIN_OPTIONS` (2) and `MAX_OPTIONS` (6) entries. This check, implemented in lines **176‑185**, prevents true/false-only questions while limiting multiple-choice complexity to manageable bounds.

### Correct Answer Index Verification (L009)

The `correct` field must be an integer index that falls within the bounds of the `options` list. Validated in lines **186‑194**, this rule prevents out-of-range errors where the correct answer points to a non-existent option.

## Content Integrity Checks

Beyond structural validation, the auditor verifies internal content consistency.

### Internal Link Resolution (L010)

Every relative Markdown link in [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must resolve to an existing file within the repository. Lines **196‑212** (note: the analysis mentions 196-112 but likely means 196-212) verify that cross-references between lessons remain valid and that no broken links enter the curriculum.

## Valid File Examples

The following examples satisfy all validation rules enforced by the audit system.

**Valid [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) structure:**

```json
{
  "questions": [
    {
      "stage": "pre",
      "question": "What is the purpose of a learning rate?",
      "options": ["Control step size", "Initialize weights", "Define loss", "Set batch size"],
      "correct": 0,
      "explanation": "The learning rate determines how large each update step is."
    },
    {
      "stage": "check",
      "question": "Select the activation that is always positive.",
      "options": ["ReLU", "Sigmoid", "Tanh"],
      "correct": 0,
      "explanation": "ReLU outputs max(0, x), never negative."
    }
  ]
}

```

**Valid [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) frontmatter:**

```markdown

# Building a Simple Neural Network

> Learn to implement a feed‑forward network from scratch.

**Type:** Build
**Languages:** python
**Prerequisites:** None
**Time:** ~30 minutes

## Learning Objectives

- Write a forward pass
- Compute gradients manually
- Train on a toy dataset

```

## Summary

- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** implements ten validation rules (L001-L010) that enforce curriculum consistency across the `rohitg00/ai-engineering-from-scratch` repository.
- Lesson directories must follow the `NN-slug` naming pattern, and [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) must exceed 200 bytes with a valid H1 header.
- **[`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json)** must be syntactically valid, use the canonical schema (not legacy keys `q`, `choices`, or `answer`), and contain 2-6 options per question with a valid `correct` index.
- The `code/` directory cannot be empty, and all internal Markdown links must resolve to existing files.
- These checks run across `phases/*/*/` directories to ensure machine-readable, consistent educational content.

## Frequently Asked Questions

### What happens if quiz.json contains legacy keys like "q" or "choices"?

The audit fails with a schema violation error. Rule **L007** (lines **157‑166**) explicitly prohibits legacy keys including `q`, `choices`, and `answer`, requiring migration to the canonical keys `question`, `options`, `correct`, and `explanation`.

### What is the required format for lesson directory names?

Lesson directories must match the regex `^[0-9]{2}-[a-z0-9][a-z0-9-]*[a-z0-9]$`, meaning they start with two digits, followed by a hyphen and a lowercase alphanumeric slug. This **L001** rule ensures consistent sorting and URL-safe identifiers.

### How many options must a quiz question include?

Each question must include between 2 and 6 options inclusive. Rule **L008** enforces these bounds through `MIN_OPTIONS` (2) and `MAX_OPTIONS` (6) constants, preventing trivial true/false questions while limiting cognitive load.

### What are the documentation size requirements for docs/en.md?

According to rule **L003** (lines **103‑114**), the [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file must be at least 200 bytes (`MIN_DOC_BYTES`). This minimum size prevents empty stubs while allowing concise lessons that still include necessary frontmatter and content structure.