AI Engineering Curriculum quiz.json Schema Requirements: Canonical vs. Legacy

The rohitg00/ai-engineering-from-scratch curriculum enforces a single canonical quiz.json schema requiring exactly six questions—one pre-assessment, three comprehension checks, and two post-assessment questions—while legacy formats using nested q/choices/answer structures are explicitly unsupported and will cause silent UI failures.

The quiz.json file serves as the assessment backbone for every lesson in the AI Engineering from Scratch repository. According to the official specification documented in AGENTS.md (source lines 88‑105), the curriculum maintains strict validation rules to ensure consistent quiz rendering across all phases and Claude certification modules.

Canonical quiz.json Schema Structure

The canonical schema defines a flat, predictable structure that the site generator consumes directly. This specification eliminates ambiguity and prevents runtime parsing errors during the build process.

Required Fields and Types

Every quiz.json file must contain these top-level and nested fields:

• lesson – String representing the directory slug (e.g., 10-llms-from-scratch)
• title – Human-readable lesson title
• questions – Array containing exactly six objects, organized by assessment stage
• stage – Enumerated string: "pre", "check", or "post" indicating when the question appears
• question – String containing the prompt text
• options – Array of exactly four strings (["a","b","c","d"])
• correct – Zero-indexed integer (0‑3) pointing to the correct option
• explanation – String providing the rationale (may be empty)

Fixed Question Distribution Schema

The curriculum enforces a rigid sequence to support pedagogical flow. The questions array must follow this exact distribution:

1. One pre-assessment question ("stage": "pre") – Establishes baseline knowledge
2. Three comprehension checks ("stage": "check") – Validates understanding during the lesson
3. Two post-assessment questions ("stage": "post") – Confirms retention after completion

Deviation from this 1‑3‑2 distribution causes validation failures in the CI pipeline.

Strict Validation Constraints

The repository implements several non-negotiable constraints that trigger build errors if violated.

Zero-indexed correct answers. The correct field uses zero-based indexing where 0 maps to the first option in the options array. Using one-based indexing (1‑4) causes incorrect scoring without warning.

Fixed array lengths. The options array must contain exactly four strings, and the questions array must contain exactly six objects. The site/build.js renderer expects these cardinalities when generating the static site.

Enumerated stage values. Only "pre", "check", and "post" are valid for the stage field. Any other string renders the quiz invisible in the generated UI.

Legacy Schema Format (Unsupported)

Earlier iterations of the curriculum experimented with an alternative JSON structure utilizing nested objects with q, choices, and answer keys. This legacy format is now explicitly unsupported.

As implemented in site/build.js, the static site generator parses only the canonical flat schema. Attempting to use the legacy nested structure results in silent failures where quizzes fail to render entirely, leaving lesson pages without assessment components. The build pipeline does not provide backward compatibility or migration warnings for legacy files.

Valid quiz.json Example

Below is a production-ready quiz.json that satisfies all canonical requirements for the tokenizers lesson:

{
  "lesson": "10-llms-from-scratch",
  "title": "Mini‑GPT Pre‑training",
  "questions": [
    {
      "stage": "pre",
      "question": "What does a token represent?",
      "options": ["A word", "A character", "A sub‑word", "A sentence"],
      "correct": 2,
      "explanation": "Tokens are typically sub‑word units."
    },
    {
      "stage": "check",
      "question": "Which loss function is used for language modeling?",
      "options": ["MSE", "Cross‑entropy", "Huber", "KL‑divergence"],
      "correct": 1,
      "explanation": "Cross‑entropy measures the probability of the actual token."
    },
    {
      "stage": "check",
      "question": "What is the purpose of positional embeddings?",
      "options": ["Encode token identity", "Encode token order", "Encode token frequency", "Encode token type"],
      "correct": 1,
      "explanation": "They provide the model with sequence order information."
    },
    {
      "stage": "check",
      "question": "Which optimizer is recommended for large‑scale training?",
      "options": ["SGD", "Adam", "RMSprop", "AdaGrad"],
      "correct": 1,
      "explanation": "Adam works well for noisy gradients."
    },
    {
      "stage": "post",
      "question": "How does temperature affect sampling?",
      "options": ["Higher → more deterministic", "Higher → more random", "Lower → more random", "Lower → no effect"],
      "correct": 1,
      "explanation": "Increasing temperature flattens the distribution, adding randomness."
    },
    {
      "stage": "post",
      "question": "What is the typical batch size for GPU training?",
      "options": ["1", "8", "32", "128"],
      "correct": 2,
      "explanation": "32 balances memory usage and gradient stability."
    }
  ]
}

This example demonstrates the required 1‑3‑2 stage distribution and zero-indexed correct answers. Files matching this pattern reside in lesson directories such as phases/10-llms-from-scratch/01-tokenizers/quiz.json.

Implementation and Enforcement

The site/build.js module serves as the official consumer of the quiz.json schema. During the build process, it validates the JSON structure and transforms valid files into rendered HTML quiz components.

The repository's CI pipeline automatically validates quiz.json files when contributors submit new lessons or updates. Any deviation from the canonical schema—including legacy formats or incorrect question counts—prevents successful deployment.

Summary

• The canonical quiz.json schema requires exactly six questions organized as 1 pre, 3 check, and 2 post stages.
• Legacy formats using nested q/choices/answer structures are unsupported and cause silent UI failures.
• The correct field uses zero-based indexing (0‑3) mapping to the four-element options array.
• AGENTS.md lines 88‑105 provides the official specification, while site/build.js enforces it during site generation.
• All lessons—including phase content and Claude certification modules—must comply with this strict schema.

Frequently Asked Questions

What happens if I use the legacy quiz.json format?

The site generator (site/build.js) will fail to parse the file, resulting in a silent failure where the quiz simply does not appear on the lesson page. No error message is displayed to the end user, but the assessment functionality will be completely absent.

How many questions must a quiz.json file contain?

The canonical schema requires exactly six questions: one pre-assessment ("pre"), three comprehension checks ("check"), and two post-assessment questions ("post"). This fixed distribution supports the curriculum's pedagogical structure.

Where is the canonical quiz.json schema defined?

The official specification resides in AGENTS.md at lines 88‑105. This document serves as the single source of truth for contributors implementing quiz files.

Is the correct answer field zero-indexed or one-indexed?

The correct field uses zero-based indexing, meaning valid values are integers 0 through 3. A value of 0 indicates the first option in the options array is correct, while 3 indicates the fourth option. Using one-based indexing (1‑4) will cause incorrect answer scoring.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}