Understanding the quiz.json Schema: Pre, Check, and Post Stages in AI Engineering From Scratch

The quiz.json schema in the ai-engineering-from-scratch repository enforces a rigid six-question structure with one pre-lesson baseline, three mid-lesson comprehension checks, and two post-lesson retention assessments.

The curriculum at rohitg00/ai-engineering-from-scratch uses structured JSON assessment files to evaluate learner progress throughout each lesson. Understanding the quiz.json schema structure is essential for contributors creating new lessons or modifying existing assessments. The schema is strictly defined in the repository's AGENTS.md file and requires exactly six questions distributed across three distinct temporal stages.

Top-Level Structure

Each lesson directory contains a single quiz.json file that must conform to a rigid top-level structure. The file defines the lesson identifier, display title, and the questions array.

The required top-level fields are:

• lesson – A string slug matching the lesson directory name (e.g., 01-dev-environment)
• title – Human-readable lesson title displayed in the UI
• questions – An array containing exactly six question objects

The site renderer validates this structure strictly during the build process. Any file deviating from the six-question requirement or missing required fields is ignored entirely.

Question Object Properties

Each object within the questions array must contain five specific properties that define the assessment item's content, timing, and correct answer.

• stage – String value indicating when to display the question: "pre", "check", or "post"
• question – String containing the prompt text shown to the learner
• options – Array of exactly four strings representing multiple-choice answers
• correct – Integer using zero-based indexing (values 0-3) indicating the position of the correct option
• explanation – String providing the rationale shown after the learner selects an answer

The correct field must reference a valid index within the options array. For example, if the correct answer is the first option listed, correct must be set to 0, not 1.

The Three Assessment Stages

The curriculum employs a distributed assessment model where questions appear at specific pedagogical moments. The schema mandates an exact distribution across three stages: one pre-stage question, three check-stage questions, and two post-stage questions.

Pre-Stage Questions

The pre-stage contains exactly one question administered before the learner begins the lesson content. This baseline assessment gauges prior knowledge and sets context for the upcoming material.

{
  "stage": "pre",
  "question": "What is the purpose of a virtual environment?",
  "options": ["Speed up code", "Isolate dependencies", "Enable GPU access", "Replace pip"],
  "correct": 1,
  "explanation": "Virtual environments keep package versions separate per project."
}

Check-Stage Questions

The check-stage contains exactly three questions interspersed throughout the lesson to verify ongoing comprehension. These formative assessments appear at logical breaking points in the content to prevent learner drift and ensure understanding before proceeding.

Each check question follows the same object structure but must specify "stage": "check". The three questions in this stage represent the bulk of the formative assessment within the lesson.

Post-Stage Questions

The post-stage contains exactly two questions administered after the lesson concludes. These summative assessments measure knowledge retention and reinforce key concepts from the completed section.

The strict requirement of two post-lesson questions ensures consistent evaluation density across all lessons in the curriculum, as enforced by the renderer.

Complete Implementation Example

Here is a minimal valid quiz.json file demonstrating all three stages with the required six-question distribution:

{
  "lesson": "example-lesson",
  "title": "Example Lesson",
  "questions": [
    {
      "stage": "pre",
      "question": "What is the purpose of a virtual environment?",
      "options": ["Speed up code", "Isolate dependencies", "Enable GPU access", "Replace pip"],
      "correct": 1,
      "explanation": "Virtual environments keep package versions separate per project."
    },
    {
      "stage": "check",
      "question": "Which command checks PyTorch GPU availability?",
      "options": ["torch.cuda.is_available()", "torch.version()", "torch.cuda()", "torch.gpu()"],
      "correct": 0,
      "explanation": "`torch.cuda.is_available()` returns true if a CUDA‑enabled GPU is reachable."
    },
    {
      "stage": "check",
      "question": "What does `uv` replace in Python projects?",
      "options": ["conda", "pip", "setuptools", "wheel"],
      "correct": 1,
      "explanation": "`uv` is a fast alternative to `pip`."
    },
    {
      "stage": "check",
      "question": "Which layer of the environment stack is installed first?",
      "options": ["AI libraries", "Package managers", "Language runtimes", "System foundation"],
      "correct": 3,
      "explanation": "Installation proceeds bottom‑up, starting with the system foundation."
    },
    {
      "stage": "post",
      "question": "After finishing the lesson, which command confirms GPU access?",
      "options": ["nvidia-smi", "torch.cuda.is_available()", "pip list", "uv help"],
      "correct": 1,
      "explanation": "The same `torch.cuda.is_available()` check is used post‑lesson."
    },
    {
      "stage": "post",
      "question": "Why are three check‑stage questions required?",
      "options": ["To match the schema", "For better grading", "Because they are optional", "To increase difficulty"],
      "correct": 0,
      "explanation": "The curriculum mandates exactly three check questions."
    }
  ]
}

Validation and Schema Enforcement

The schema definition resides in AGENTS.md at the repository root, specifically within the section titled quiz.json schema. The site renderer parses every quiz.json file found under phases/**/ and validates against these strict requirements.

Files that deviate from the six-question rule, use incorrect stage values, or contain improperly indexed correct answers are rejected silently during the build process. For a production-ready example, examine phases/00-setup-and-tooling/01-dev-environment/quiz.json, which implements the exact schema structure documented above for the Dev Environment lesson.

Summary

• The quiz.json schema requires exactly six questions per lesson file to pass validation.
• Questions distribute as one pre-stage, three check-stage, and two post-stage assessments.
• Each question object must include stage, question, options (4 items), correct (0-based index), and explanation properties.
• The schema is enforced by the site renderer according to specifications in AGENTS.md.
• Real-world implementations follow this structure strictly, as seen in phases/00-setup-and-tooling/01-dev-environment/quiz.json.

Frequently Asked Questions

What happens if a quiz.json file contains fewer than six questions?

The site renderer will ignore the file entirely during the build process. The schema mandates exactly six questions to maintain consistent assessment density across all lessons in the curriculum, and any deviation causes validation failure.

Can I use a different number of options per question?

No. Each question must contain exactly four options in the options array. The correct field must reference one of these four options using zero-based indexing (values 0 through 3), as enforced by the renderer.

Where is the quiz.json schema officially defined?

The authoritative schema definition lives in AGENTS.md at the root of the rohitg00/ai-engineering-from-scratch repository, under the section labeled quiz.json schema. This document serves as the source of truth for the site renderer's validation logic.

Are the check-stage questions randomized or fixed in position?

According to the source code, the three check-stage questions are interspersed throughout the lesson at specific pedagogical points, but they follow the fixed order defined in the questions array within the JSON file itself. The schema does not support randomization of the stage order.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}