# How Schema Validation for quiz.json Files Works in AI-Engineering-From-Scratch

> Learn how quiz.json schema validation works in AI-Engineering-From-Scratch. Discover how the audit script enforces rules, rejects old formats, and validates settings.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-18

---

**The AI-Engineering-From-Scratch repository validates quiz.json files through a centralized audit script located at [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), which enforces canonical schema rules, rejects legacy key formats, and validates option arrays and correct answer indices.**

The `ai-engineering-from-scratch` curriculum maintains strict data integrity through automated schema validation for quiz.json files. Located in the [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) utility, this validation system ensures every lesson's quiz adheres to a canonical structure before passing continuous integration checks. The audit process validates JSON parsing, top-level data structures, per-question field requirements, and logical consistency across all question data.

## The Audit Entry Point: [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)

The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) file serves as the central validation driver for the entire curriculum. According to the source code, this script runs as part of the "audit" step that checks all lessons for compliance with curriculum rules. When the script encounters a [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file in any lesson directory (typically located under `phases/*/*/quiz.json`), it triggers a comprehensive validation pipeline that reports specific error codes (L006-L009) when violations occur.

## JSON Parsing and Top-Level Structure Validation

### Parsing and Decoding Checks

The validation begins with strict JSON parsing. In [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) lines 33-38, the script attempts to read and decode the file content using `json.loads()`. If the file contains malformed JSON, syntax errors, or encoding issues, the audit immediately reports an **L006** issue and halts processing for that specific lesson.

### Acceptable Root Schema Types

Following successful parsing, lines 39-45 in [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) validate the top-level data structure. The schema accepts two valid formats:

- **Direct array format**: A JSON list where each element is a question object
- **Wrapped object format**: A JSON dictionary containing a non-empty `"questions"` key that holds the question array

Any other structure triggers an **L006** issue, ensuring consistent data shapes across the curriculum.

## Canonical Quiz Schema Definition

The expected schema is defined through two critical constants declared in lines 30-33 of [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py):

```python
CANONICAL_QUIZ_KEYS = {"stage", "question", "options", "correct", "explanation"}
LEGACY_QUIZ_KEYS = {"q", "choices", "answer"}
MIN_OPTIONS, MAX_OPTIONS = 2, 6

```

These constants establish the contract for valid quiz data. The **canonical keys** represent the current standard, while **legacy keys** trigger deprecation warnings. The options boundaries ensure every question contains between 2 and 6 possible answers.

## Per-Question Validation Rules

The core validation logic spans lines 53-94 in [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py), iterating through each question in the array and applying four distinct checks:

### Legacy Key Detection (L007)

If any question contains deprecated keys (`q`, `choices`, or `answer`), the audit reports an **L007** issue. This enforcement prevents drift toward outdated schema formats and ensures all content uses the modern canonical structure.

### Required Field Validation (L006)

Every question must contain all five canonical keys: `stage`, `question`, `options`, `correct`, and `explanation`. The validation computes the set difference between `CANONICAL_QUIZ_KEYS` and the actual question keys. Missing fields generate an **L006** issue with specific details about which keys are absent.

### Options Array Constraints (L008)

The `options` field must be a list with a length between `MIN_OPTIONS` (2) and `MAX_OPTIONS` (6). Violations of this range produce an **L008** issue, ensuring consistent user experience across all quiz questions.

### Correct Answer Index Verification (L009)

The `correct` field must be an integer that points to a valid zero-based index within the `options` array. If the value is negative or exceeds the length of the options list, the audit raises an **L009** issue. This prevents runtime errors where the correct answer points to a non-existent option.

## Error Codes and CI Integration

The validation system uses a structured error code taxonomy:

- **L006**: JSON parsing failures, invalid top-level structures, or missing required keys
- **L007**: Usage of deprecated legacy keys
- **L008**: Options array length violations (fewer than 2 or more than 6 items)
- **L009**: Invalid correct answer indices

When any check fails, the script reports the specific error code, lesson path, and detailed diagnostic message, then aborts the CI check for that lesson. This prevents corrupted or malformed quiz data from merging into the main branch.

## Summary

- **Validation Location**: All quiz.json validation occurs in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)
- **Structure Requirements**: Files must contain either a question array or an object with a `"questions"` key
- **Canonical Fields**: Valid questions require `stage`, `question`, `options`, `correct`, and `explanation`
- **Legacy Migration**: Deprecated keys (`q`, `choices`, `answer`) trigger L007 warnings
- **Option Constraints**: Questions must have 2-6 options (L008) with a valid index in `correct` (L009)
- **Error Reporting**: The system uses codes L006-L009 to categorize specific validation failures

## Frequently Asked Questions

### What happens if a quiz.json file contains invalid JSON?

The audit script attempts to parse the file using `json.loads()` in lines 33-38 of [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py). If parsing fails due to malformed syntax or encoding errors, the system immediately reports an **L006** issue and stops processing that specific lesson, preventing the invalid file from passing CI checks.

### What is the difference between canonical and legacy quiz keys?

Canonical keys represent the current schema standard: `stage`, `question`, `options`, `correct`, and `explanation`. Legacy keys (`q`, `choices`, `answer`) are deprecated formats from earlier versions of the curriculum. If [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) detects any legacy keys in a question, it raises an **L007** issue to enforce migration to the modern schema.

### How many options must a quiz question have?

Each question must contain between 2 and 6 options, as defined by the `MIN_OPTIONS` and `MAX_OPTIONS` constants (set to 2 and 6 respectively) in lines 30-33 of [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py). Violations of these bounds generate an **L008** error during the audit process.

### Where is the quiz validation logic located?

The primary validation logic resides in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), specifically in lines 53-94, which contain the per-question validation loop. This script checks JSON structure, key presence, options array length, and correct answer indices for every lesson's [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file located under the `phases/` directory structure.