quiz.json Schema for AI Engineering From Scratch: Exact Structure and Validation Rules

The quiz.json schema in rohitg00/ai-engineering-from-scratch requires a top-level object containing lesson, title, and questions fields, where questions must be a fixed-length array of exactly six objects distributed across pre, check, and post stages.

The ai-engineering-from-scratch curriculum uses a standardized quiz.json format to deliver consistent assessments across every lesson. Understanding the exact quiz.json schema is essential for contributors writing new lessons or maintaining existing content, as the repository enforces strict validation through automated CI checks that reject any deviation from the canonical structure defined in AGENTS.md.

Top-Level Structure

Each lesson directory contains a quiz.json file located at phases/*/*/quiz.json. The top-level object contains exactly three fields:

• lesson – The slug of the lesson directory (e.g., 87-end-to-end-safety-gate).
• title – A human-readable title describing the lesson content.
• questions – An array containing exactly six question objects.

The repository's validation scripts (scripts/audit_lessons.py) enforce this precise count, and the site renderer (site/build.js) expects this shape; any deviation will cause the quiz to fail silently or break the build pipeline.

Stage Distribution Requirements

The six questions must follow a rigid order and stage distribution to align with the curriculum's learning flow:

Index	Stage	Purpose
0	pre	A warm-up question shown before any code is run.
1–3	check	Three mid-lesson checks verifying comprehension of core material.
4–5	post	Two wrap-up questions reinforcing learning after lesson completion.

This 1-3-2 distribution is hardcoded in the validation logic. The stage field categorizes each question's timing in the learning flow and must be one of the three string literals: "pre", "check", or "post".

Question Object Schema

Each object in the questions array must contain the following keys, as formally defined in AGENTS.md (lines 83–99):

• stage – String enum: "pre", "check", or "post".
• question – String containing the multiple-choice prompt presented to the learner.
• options – Array of exactly four answer strings (e.g., ["Outputs Hello","Raises error","Does nothing","Creates a file"]).
• correct – Integer from 0 to 3 indicating the zero-based index of the correct option in the options array.
• explanation – Optional string providing rationale shown after the learner answers, useful for clarifying why a particular choice is right or wrong.

Validation and Enforcement

The schema is strictly enforced across the entire curriculum through multiple mechanisms:

AGENTS.md (lines 83–99) serves as the canonical documentation source, defining the formal schema that all contributors must follow.

scripts/audit_lessons.py runs in CI to validate that every quiz.json file contains exactly six questions with the correct stage distribution and required fields.

site/build.js consumes these JSON files to generate the curriculum UI, expecting the precise shape described above; legacy formats like {q, choices, answer} are not supported and will cause rendering failures.

Working with quiz.json

Below is a minimal valid quiz.json for a hypothetical lesson:

{
  "lesson": "01-hello-world",
  "title": "Hello World",
  "questions": [
    { "stage": "pre",   "question": "What does `print('Hello')` do?", "options": ["Outputs Hello","Raises error","Does nothing","Creates a file"], "correct": 0, "explanation": "It prints the string." },
    { "stage": "check", "question": "Which Python version introduced f‑strings?", "options": ["3.5","3.6","3.7","3.8"], "correct": 1, "explanation": "f‑strings arrived in 3.6." },
    { "stage": "check", "question": "Which package is allowed for numeric work?", "options": ["numpy","pandas","scikit‑learn","tensorflow"], "correct": 0, "explanation": "Only `numpy` (and other allow‑list libs) are permitted." },
    { "stage": "check", "question": "What is the output of `len([1,2,3])`?", "options": ["2","3","4","Error"], "correct": 1, "explanation": "Length of a three‑element list is 3." },
    { "stage": "post",  "question": "Can you use `torch` for this lesson?", "options": ["Yes","No"], "correct": 1, "explanation": "`torch` is allowed only where explicitly needed." },
    { "stage": "post",  "question": "Is the lesson complete?", "options": ["Yes","No"], "correct": 0, "explanation": "All steps passed." }
  ]
}

To programmatically load and validate a quiz file in Python:

import json
import pathlib

quiz_path = pathlib.Path(__file__).parent / "quiz.json"
quiz = json.loads(quiz_path.read_text())

assert len(quiz["questions"]) == 6, "Exactly six questions required"
pre_questions = [q for q in quiz["questions"] if q["stage"] == "pre"]
assert len(pre_questions) == 1, "One pre‑question expected"

Summary

• The quiz.json schema requires exactly three top-level fields: lesson, title, and questions.
• The questions array must contain exactly six objects following a strict 1-3-2 distribution across pre, check, and post stages.
• Each question must include stage, question, options (4 items), correct (0-3), and optional explanation fields.
• Validation occurs via scripts/audit_lessons.py and the canonical definition lives in AGENTS.md (lines 83–99).

Frequently Asked Questions

What happens if a quiz.json file doesn't have exactly six questions?

The CI validation script (scripts/audit_lessons.py) will reject the lesson, and the site builder (site/build.js) may fail to render the quiz or display errors in the curriculum UI. The repository strictly enforces the six-question requirement to maintain consistency across all learning modules.

Can I use a legacy format like {q, choices, answer} for backward compatibility?

No. The repository does not support legacy formats. The site renderer expects the current schema with stage, question, options, correct, and explanation keys. Using deprecated field names will cause the quiz to fail silently or break the lesson display.

Where is the official quiz.json schema defined in the repository?

The canonical schema definition resides in AGENTS.md at lines 83–99. This document serves as the single source of truth for the quiz.json structure, question field requirements, and stage distribution rules used by both validators and the site renderer.

How does the curriculum site render quiz data from these JSON files?

The site/build.js script consumes the quiz.json files from each lesson directory (phases/*/*/quiz.json) and transforms them into the interactive quiz components displayed in the curriculum UI. It relies on the fixed schema to determine when to show pre-lesson warm-ups, mid-lesson checks, and post-lesson reviews.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}