# How the quiz.json Schema Works in AI Engineering From Scratch: Stage Requirements and Validation Rules

> Understand the quiz.json schema in ai-engineering-from-scratch. Learn about stage requirements and validation rules for AI quiz questions.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: architecture
- Published: 2026-06-14

---

**The [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file in the ai-engineering-from-scratch repository follows a strict schema requiring exactly six questions per lesson—one `pre`-stage, three `check`-stage, and two `post`-stage questions—each containing `question`, `options` (four choices), `correct` (zero-based index), and `explanation` fields.**

The ai-engineering-from-scratch curriculum uses a standardized [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) schema to ensure consistent assessment across all lessons. Each quiz file must adhere to specific structural requirements and question stage distributions defined in the repository's [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md) contract. Understanding this schema is essential for contributors adding new lessons or modifying existing assessments.

## Understanding the quiz.json Schema Structure

### Top-Level Properties

Every [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file must contain three top-level keys:

- **`lesson`**: A string representing the directory slug (e.g., `13-question-answering`)
- **`title`**: A human-readable string describing the lesson
- **`questions`**: An array containing exactly six question objects

### Question Object Fields

Each question object within the `questions` array requires five specific fields:

| Field | Type | Description |
|-------|------|-------------|
| `stage` | string | One of `pre`, `check`, or `post` |
| `question` | string | The prompt presented to the learner |
| `options` | array | Exactly four answer choice strings |
| `correct` | integer | Zero-based index (0-3) of the correct answer |
| `explanation` | string | Rationale for the correct answer (can be empty) |

## Question Stage Requirements

The curriculum uses three distinct stages to assess learning progression. The `stage` field determines when a question appears in the learner's journey, and the counts are strictly enforced by the validator.

### Pre-Stage Questions

**Pre-stage** questions appear before the learner engages with lesson content to gauge prior knowledge. Exactly one pre-stage question is required per quiz. These establish a baseline for measuring learning gains.

### Check-Stage Questions

**Check-stage** questions are interspersed throughout the lesson to verify immediate comprehension of the material just presented. The schema requires exactly three check-stage questions, allowing for multiple knowledge checkpoints during the lesson flow.

### Post-Stage Questions

**Post-stage** questions appear after the learner completes the lesson to measure retained knowledge and overall understanding. Each quiz must include exactly two post-stage questions to assess learning outcomes.

## Creating a Valid quiz.json File

Below is a complete example showing how to generate a properly structured [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file that satisfies all stage requirements:

```python
import json

quiz = {
    "lesson": "99-sample-lesson",
    "title": "Sample Lesson Title",
    "questions": [
        # Exactly 1 pre-stage question

        {
            "stage": "pre",
            "question": "What do you already know about this topic?",
            "options": ["Nothing", "Basics", "Intermediate", "Expert"],
            "correct": 0,
            "explanation": "This is a diagnostic question with no wrong answer."
        },
        # Exactly 3 check-stage questions

        {
            "stage": "check",
            "question": "Which algorithm is best for this task?",
            "options": ["Linear Regression", "BERT", "K-Means", "Decision Tree"],
            "correct": 1,
            "explanation": "BERT is specifically designed for NLP tasks."
        },
        {
            "stage": "check",
            "question": "What does RAG stand for?",
            "options": ["Random Access Grade", "Retrieval-Augmented Generation", "Recursive Algorithm Graph", "None of the above"],
            "correct": 1,
            "explanation": "RAG combines retrieval systems with generative models."
        },
        {
            "stage": "check",
            "question": "Which metric evaluates QA systems?",
            "options": ["Accuracy", "BLEU", "F1 Score", "All of the above"],
            "correct": 3,
            "explanation": "Multiple metrics are used depending on the task."
        },
        # Exactly 2 post-stage questions

        {
            "stage": "post",
            "question": "When should you use fine-tuning vs RAG?",
            "options": ["Always fine-tune", "Always RAG", "Depends on data", "Neither works"],
            "correct": 2,
            "explanation": "The choice depends on your specific data and latency requirements."
        },
        {
            "stage": "post",
            "question": "What is the main bottleneck in semantic search?",
            "options": ["Embedding generation", "Vector database latency", "Query parsing", "All of the above"],
            "correct": 1,
            "explanation": "Vector database latency often dominates retrieval time."
        }
    ]
}

with open("quiz.json", "w") as f:
    json.dump(quiz, f, indent=2)

```

## Validating quiz.json Against the Schema

The repository includes validation logic in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) to enforce the schema during CI checks. You can implement a standalone validator to check stage counts before committing:

```python
import json
import collections
import pathlib

def validate_quiz(path: pathlib.Path):
    """Validate quiz.json stage distribution and structure."""
    data = json.loads(path.read_text())
    
    assert "questions" in data, "Missing questions list"
    assert len(data["questions"]) == 6, "Must have exactly 6 questions"
    
    stages = collections.Counter(q["stage"] for q in data["questions"])
    required = {"pre": 1, "check": 3, "post": 2}
    
    if stages != required:
        raise ValueError(f"Invalid stage distribution: {stages}. Required: {required}")
    
    for i, q in enumerate(data["questions"]):
        assert 0 <= q["correct"] < len(q["options"]), \
            f"Question {i}: correct index out of range"
        assert len(q["options"]) == 4, \
            f"Question {i}: must have exactly 4 options"
        assert "explanation" in q, \
            f"Question {i}: missing explanation field"
    
    print("✅ Quiz validation passed")
    return data

# Usage

if __name__ == "__main__":
    validate_quiz(pathlib.Path("quiz.json"))

```

## Key Implementation Files

Understanding the [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) schema requires familiarity with these specific files in the rohitg00/ai-engineering-from-scratch repository:

- **[`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md)**: Defines the complete curriculum contract, including the exact [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) schema specifications and stage requirements
- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)**: CI helper that validates every [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) against the schema during pull request checks
- **[`phases/05-nlp-foundations-to-advanced/13-question-answering/quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/05-nlp-foundations-to-advanced/13-question-answering/quiz.json)**: Reference implementation showing a fully populated quiz with proper pre-, check-, and post-stage distribution

## Summary

- **Exactly six questions** are required per lesson: one `pre`-stage, three `check`-stage, and two `post`-stage questions
- **Four options** must be provided for each question, with the `correct` field containing a zero-based index (0-3)
- **Explanation field** is required by the validator but may contain an empty string if preferred
- **File location** must be [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) in the lesson directory (e.g., `phases/<phase>/<lesson>/quiz.json`)
- **Validation** is enforced automatically by [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) in the CI pipeline

## Frequently Asked Questions

### What happens if I include the wrong number of questions or stages?

The site renderer will reject the quiz, and the CI pipeline will fail. The validator in [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) specifically checks that the stage distribution matches exactly one `pre`, three `check`, and two `post` questions. Any deviation triggers a `ValueError` during the audit process.

### Can I reorder the questions in the array?

Yes, the ordering of question objects within the `questions` array does not affect validation. However, the stage counts must still satisfy the 1-3-2 distribution regardless of position. The curriculum platform may render questions in the order they appear, so logical grouping by stage is recommended for readability.

### Is the explanation field truly optional?

While the `explanation` field must be present in every question object (the validator checks for its existence), the string content can be empty (`""`). This allows contributors to include explanatory text where helpful while maintaining schema compliance for questions where explanations are unnecessary.

### How does the CI validate quiz.json files automatically?

The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) script traverses all lesson directories under `phases/` and validates each [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) against the schema. It checks for JSON syntax errors, ensures exactly six questions exist, verifies the stage distribution counters, and validates that the `correct` index falls within the bounds of the `options` array. Failures block pull request merges.