How Quizzes Are Structured for Each Lesson in the AI Engineering from Scratch Course

Each lesson in the AI Engineering from Scratch curriculum ships a self-contained quiz.json file containing exactly six questions divided into three stages: one pre-lesson warm-up, three comprehension checks, and two post-lesson reinforcement questions.

The rohitg00/ai-engineering-from-scratch repository organizes every lesson around a deterministic quiz schema defined in AGENTS.md. This structure ensures consistent assessment across all phases of the curriculum, from math foundations to LLM implementation. The six-question rule is strictly enforced by CI pipelines to maintain compatibility with the learning platform and interactive agents.

The quiz.json Schema

Every lesson directory contains a quiz.json file that follows a rigid schema. According to the specification in AGENTS.md (lines 90-106), the file must contain a questions array with objects adhering to specific field requirements.

Six-Question Layout

The curriculum mandates exactly six questions per lesson, distributed across three stages:

• 1 pre question: Warm-up or sanity check before the lesson begins
• 3 check questions: Core comprehension checks while studying the material
• 2 post questions: Reinforcement and retention testing after completion

This 1-3-2 distribution is hardcoded into the validation logic. Any deviation causes the lesson quiz to be ignored by the site renderer and learning agents.

Required Fields

Each question object in the questions array must include:

• stage: String value of "pre", "check", or "post"
• question: String containing the multiple-choice question text
• options: Array of exactly four answer strings
• correct: Zero-based integer index indicating the correct option
• explanation: String providing the rationale after answer submission

Optional fields include lesson (directory slug) and title (human-readable lesson name) for tooling integration.

Validation and Enforcement

The repository maintains strict schema compliance through automated audits and build-time checks.

CI Audit Scripts

The scripts/audit_lessons.py file validates every quiz.json in the repository. This script:

• Verifies the six-question count (1 pre + 3 check + 2 post)
• Confirms all required fields are present
• Validates that options arrays contain exactly four items
• Checks that correct indices fall within the 0-3 range

Failures in this audit block CI pipelines, preventing malformed quizzes from reaching learners.

Site Builder Integration

The site/build.js process relies on the standardized schema to render quizzes without external resources. Because questions carry complete text, options, correct answers, and explanations internally, the platform can present them deterministically without dynamic computation or database queries.

Real-World Examples

The schema applies uniformly across the curriculum, from mathematical foundations to advanced LLM concepts.

Math Foundations Lesson

The linear algebra intuition lesson at phases/01-math-foundations/01-linear-algebra-intuition/quiz.json demonstrates the schema in practice. Its pre question asks about vector dot products, while post questions cover matrix rank in machine learning contexts. This structure ensures learners grasp geometric intuition before advancing to computational applications.

LLM Tokenizer Lesson

Similarly, the tokenizer lesson at phases/10-llms-from-scratch/01-tokenizers/quiz.json follows the identical six-question pattern. Despite covering advanced NLP concepts, it maintains the same pre/check/post progression to scaffold learning from byte-pair encoding fundamentals to implementation details.

Working with Quiz Data Programmatically

The skills/learn-agent-skills/SKILL.md documentation describes how autonomous agents consume these quizzes. You can implement similar tooling using the repository's schema.

Loading and Filtering Questions

To extract specific question stages from a lesson quiz:

import json
from pathlib import Path

def load_quiz(quiz_path: Path) -> dict:
    """Read a quiz.json file and return the parsed object."""
    with quiz_path.open(encoding="utf-8") as f:
        return json.load(f)

def post_questions(quiz: dict) -> list[dict]:
    """Filter for questions whose stage is 'post'."""
    return [q for q in quiz["questions"] if q["stage"] == "post"]

# Example usage:

quiz_file = Path(
    "phases/01-math-foundations/01-linear-algebra-intuition/quiz.json"
)
quiz = load_quiz(quiz_file)
for idx, q in enumerate(post_questions(quiz), start=1):
    print(f"Post-question {idx}: {q['question']}")
    for i, opt in enumerate(q["options"]):
        print(f"  {i+1}. {opt}")
    print()

Validating Schema Compliance

To programmatically verify a quiz follows the required structure:

def validate_quiz_schema(quiz: dict) -> bool:
    """Return True if quiz follows exact 1-pre/3-check/2-post pattern."""
    stages = [q["stage"] for q in quiz["questions"]]
    return (
        stages.count("pre") == 1 and
        stages.count("check") == 3 and
        stages.count("post") == 2 and
        len(quiz["questions"]) == 6
    )

assert validate_quiz_schema(quiz), "Quiz does not match required schema"

These patterns mirror the validation logic found in scripts/audit_lessons.py, ensuring your tooling remains compatible with the curriculum's requirements.

Summary

• Every lesson includes a quiz.json file located in its respective phase directory (e.g., phases/01-math-foundations/01-linear-algebra-intuition/)
• Six-question mandate: Exactly 1 pre, 3 check, and 2 post questions per lesson
• Self-contained structure: Each question includes text, four options, correct index, and explanation
• Strict validation: scripts/audit_lessons.py enforces schema compliance in CI
• Agent compatibility: The format supports both human learners and autonomous agents consuming the skills/learn-agent-skills protocol

Frequently Asked Questions

How many questions are in each lesson quiz?

Each lesson quiz contains exactly six questions: one pre-lesson warm-up question, three comprehension check questions during the lesson, and two post-lesson reinforcement questions. This 1-3-2 distribution is mandatory across all phases of the curriculum.

What is the purpose of the 'pre' stage questions?

The pre stage serves as a warm-up or sanity check to assess prior knowledge before learners engage with new material. Located at stage: "pre" in the quiz.json file, this single question activates relevant mental models and signals whether the learner is prepared for the lesson's complexity.

How does the curriculum validate quiz structure?

The repository runs scripts/audit_lessons.py in CI to validate every quiz.json against the schema defined in AGENTS.md. This audit enforces the six-question rule, verifies field presence, and ensures the options array contains exactly four items. Failures prevent site deployment.

Where is the quiz schema officially documented?

The canonical schema definition resides in AGENTS.md under the "quiz.json schema" section (lines 90-106). This documentation specifies required fields, the six-question layout, and validation rules. Additional implementation details appear in skills/learn-agent-skills/SKILL.md, which describes how agents consume quiz data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}