# How `quiz.json` Schema Validation Works in the AI Engineering Curriculum

> Discover how ai-engineering-from-scratch validates quiz.json schema on every pull request. Learn about automated audit scripts enforcing constraints for robust AI curriculum.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-08-29

---

**The `ai-engineering-from-scratch` curriculum enforces strict [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) validation through automated audit scripts that check for required keys, legacy schema rejection, option count constraints, and correct index bounds on every pull request.**

The curriculum repository relies on a standardized JSON structure to power its lesson quizzes. According to the source code in `rohitg00/ai-engineering-from-scratch`, every [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file undergoes rigorous automated validation to ensure data integrity before deployment.

## Automated Validation Pipeline

The validation process runs continuously via **GitHub Actions**. The workflow defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) executes [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) for standard lessons and [`scripts/audit_certifications.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_certifications.py) for certification tracks. If any check fails, the script records a specific issue code—such as `L006` for missing keys or `L007` for legacy schema usage—and the pull request build fails, preventing malformed quizzes from merging into the main branch.

## Canonical Schema Requirements

The audit script enforces a strict key schema in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) (lines 30–67). Two constant sets define acceptable and prohibited keys:

```python
CANONICAL_QUIZ_KEYS = {"stage", "question", "options", "correct", "explanation"}
LEGACY_QUIZ_KEYS = {"q", "choices", "answer"}

```

**Legacy keys are rejected.** If the validator detects any intersection between `LEGACY_QUIZ_KEYS` and the question object’s keys, it emits error code `L007` and halts processing for that question. Conversely, if any keys from `CANONICAL_QUIZ_KEYS` are absent, the script issues error code `L006` indicating the specific missing fields.

### Root Structure Validation

The validator accepts two top-level structures (lines 39–45). The file must contain either a non-empty array of question objects directly, or a JSON object containing a non-empty `questions` array. Any other structure triggers a validation failure before individual question inspection begins.

## Question-Level Validation Rules

Once parsed, every entry in the `questions` array must pass type and content constraints implemented in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py).

### Object Type Verification

Each question must be a JSON object. The script explicitly checks `isinstance(q, dict)` (lines 53–56), issuing error `L006` if an array entry is not a dictionary.

### Option Count Constraints

The `options` field must be an array containing **2 to 6 items**. The validation logic (lines 76–84) verifies both that `options` is a list and that `2 <= len(options) <= 6`. Failure results in error code `L008`.

### Correct Index Verification

The `correct` field must be an integer index pointing to a valid position within the `options` array. The validator confirms `isinstance(correct, int)` and checks the bounds `0 <= correct < len(options)` (lines 86–93). An out-of-bounds index or non-integer value triggers error `L009`.

## Stage Structure Conventions

While the audit scripts do not programmatically enforce the exact distribution of stages, the curriculum documentation in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) specifies a six-question structure: one `pre` stage, three `check` stages, and two `post` stages. The `stage` key in each question object must contain one of these string values to maintain consistency across lesson files.

## Summary

- **Automated enforcement:** The CI pipeline runs [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) on every PR to validate [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files.
- **Strict key schema:** Questions must use the canonical keys (`stage`, `question`, `options`, `correct`, `explanation`) and cannot contain legacy aliases (`q`, `choices`, `answer`).
- **Structural rules:** Files must contain a non-empty array of questions or an object with a `questions` array.
- **Content constraints:** Questions require 2–6 options, and the `correct` field must be a valid 0-based integer index within those options.
- **Error codes:** Validation failures return specific codes (`L006`, `L007`, `L008`, `L009`) to guide authors toward precise fixes.

## Frequently Asked Questions

### What happens if I use legacy keys like "q" or "answer" in my quiz.json?

The validator detects legacy keys by checking for intersection with `LEGACY_QUIZ_KEYS` and immediately issues error code `L007`. The build fails, and the PR cannot merge until you replace legacy keys with their canonical equivalents (`question` for `q`, `options` for `choices`, `correct` for `answer`).

### How many options must each quiz question have?

Every question must contain between **2 and 6 options** inclusive. The audit script validates the length of the `options` array (lines 76–84), and any value outside this range produces error `L008`.

### What is the valid range for the "correct" index?

The `correct` value must be a zero-based integer index that falls within the bounds of the `options` array. For an array of length *n*, valid values range from `0` to `n-1`. The validator checks this constraint at lines 86–93 and reports violations as error `L009`.

### Does the validation enforce the six-question stage structure?

The automated scripts validate that the `stage` key exists and is a string, but they do not enforce the specific distribution of one `pre`, three `check`, and two `post` stages. This six-question convention is documented in [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) and expected to be followed manually by curriculum authors.