How the quiz.json Schema Manages Pre, Check, and Post Question Phases in AI Engineering From Scratch

The quiz.json schema uses a stage field with pre, check, and post values to control exactly when each question appears during the lesson lifecycle, enforcing a strict distribution of six questions per lesson (1 pre, 3 check, 2 post) validated by CI scripts.

The ai-engineering-from-scratch repository implements a structured assessment system to reinforce machine learning concepts through strategically timed quizzes. Each lesson directory contains a quiz.json file that follows a rigorous schema defined in AGENTS.md. Understanding how this quiz.json schema orchestrates pre, check, and post question phases ensures consistent pedagogical flow across the curriculum.

Schema Structure and Required Fields

Every lesson stores its interactive assessment in a quiz.json file located within the lesson folder (e.g., phases/01-math-foundations/10-dimensionality-reduction/quiz.json). The file contains a top-level questions array where each object must specify five required keys:

• stage: A string enum of pre, check, or post that dictates presentation timing
• question: The text prompt displayed to the learner
• options: An array of exactly four answer strings
• correct: A zero-based integer index (0-3) pointing to the correct option in the options array
• explanation: A rationale displayed after the learner answers, regardless of correctness

The Three Question Phases Explained

Pre-Stage Assessment

Questions marked with "stage": "pre" execute before any lesson content renders. These function as prior-knowledge diagnostics, helping learners identify gaps in foundational concepts like linear algebra or activation functions before encountering new material. The curriculum requires exactly one pre-stage question per lesson to establish a baseline without overwhelming the learner upfront.

Check-Stage Formative Questions

The check stage provides interleaved practice during the lesson, typically appearing after key concept blocks. With three check questions required per lesson, these prompts interrupt the content flow at author-defined breakpoints to reinforce learning while neural network architectures or mathematical proofs remain fresh. This immediate retrieval practice prevents misconceptions from solidifying before the lesson concludes.

Post-Stage Consolidation

Questions with "stage": "post" trigger after the lesson code finishes execution. These two questions assess holistic comprehension rather than isolated facts, verifying that the learner can integrate the full context of dimensionality reduction techniques or backpropagation mechanics. The post-stage serves as a final knowledge check before the learner progresses to the next module.

Stage Distribution Rules and CI Validation

The curriculum enforces rigid structural constraints through scripts/audit_lessons.py to maintain predictable cognitive load. Every quiz.json must contain exactly six questions distributed according to this fixed schema:

Stage	Required Count	Purpose
pre	1	Prior knowledge assessment
check	3	Mid-lesson formative checks
post	2	Post-lesson consolidation

If a lesson deviates from this 1-3-2 distribution or includes an invalid correct index outside the 0-3 range, the CI job fails with a descriptive error. This validation ensures that all lessons in the phases/ directory maintain consistent pacing and assessment density.

Architectural Flow in the Lesson Runner

When a lesson initializes, the runner script located at phases/**/code/main.* (language-specific implementations) locates and loads the sibling quiz.json using path resolution relative to the executing file:

import json
import pathlib

quiz_path = pathlib.Path(__file__).with_name('quiz.json')
questions = json.load(open(quiz_path))['questions']

The runner then partitions the questions into three staged cohorts using list comprehensions:

pre_questions   = [q for q in questions if q['stage'] == 'pre']
check_questions = [q for q in questions if q['stage'] == 'check']
post_questions  = [q for q in questions if q['stage'] == 'post']

These cohorts execute sequentially throughout the lesson lifecycle. The runner validates the distribution counts at runtime, aborting with a clear assertion error if the schema is violated. This prevents broken lessons from reaching learners even if they bypass the CI validation.

Summary

• The quiz.json schema uses a mandatory stage field with values pre, check, and post to control question presentation timing.
• Each lesson must contain exactly six questions: one pre, three check, and two post, enforced by scripts/audit_lessons.py during CI.
• The correct field uses zero-based indexing (0-3) to align with Python list access patterns in the validation and display logic.
• Lesson runners in code/main.* files partition questions by stage and orchestrate them at pedagogically strategic moments: before content, during key concepts, and after completion.
• The schema definition in AGENTS.md serves as the canonical contract between lesson authors and automated validation tools.

Frequently Asked Questions

What happens if a quiz.json file doesn't have exactly six questions?

The continuous integration pipeline will fail when scripts/audit_lessons.py detects the deviation. Additionally, the lesson runner performs runtime validation, aborting execution with a descriptive error message if the 1-3-2 stage distribution is incorrect or if the questions array length differs from six.

Why does the correct field use zero-based indexing instead of one-based?

The zero-based index aligns with Python's native list indexing conventions. Since downstream scripts treat the correct value directly as an index into the options array (e.g., selected = question['options'][question['correct']]), using 0-3 prevents off-by-one errors in the validation logic and display rendering.

Can I add more than three check-stage questions to cover complex topics?

No. The schema strictly enforces exactly three check questions per lesson to maintain consistent cognitive load and timing across the curriculum. If additional assessment is necessary, the content should be divided into multiple lessons rather than extending the quiz beyond the six-question limit defined in AGENTS.md.

Where is the canonical definition of the quiz.json schema documented?

The definitive schema specification resides in AGENTS.md under the "Lesson contract → quiz.json schema" section. This file serves as the contract between lesson authors and the automated validation tools that maintain curriculum integrity, detailing the required fields, stage distribution rules, and validation logic implemented in scripts/audit_lessons.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}