How `quiz.json` Schema Validation Works in the AI Engineering Curriculum

The ai-engineering-from-scratch curriculum enforces strict quiz.json validation through automated audit scripts that check for required keys, legacy schema rejection, option count constraints, and correct index bounds on every pull request.

The curriculum repository relies on a standardized JSON structure to power its lesson quizzes. According to the source code in rohitg00/ai-engineering-from-scratch, every quiz.json file undergoes rigorous automated validation to ensure data integrity before deployment.

Automated Validation Pipeline

The validation process runs continuously via GitHub Actions. The workflow defined in .github/workflows/curriculum.yml executes scripts/audit_lessons.py for standard lessons and scripts/audit_certifications.py for certification tracks. If any check fails, the script records a specific issue code—such as L006 for missing keys or L007 for legacy schema usage—and the pull request build fails, preventing malformed quizzes from merging into the main branch.

Canonical Schema Requirements

The audit script enforces a strict key schema in scripts/audit_lessons.py (lines 30–67). Two constant sets define acceptable and prohibited keys:

CANONICAL_QUIZ_KEYS = {"stage", "question", "options", "correct", "explanation"}
LEGACY_QUIZ_KEYS = {"q", "choices", "answer"}

Legacy keys are rejected. If the validator detects any intersection between LEGACY_QUIZ_KEYS and the question object’s keys, it emits error code L007 and halts processing for that question. Conversely, if any keys from CANONICAL_QUIZ_KEYS are absent, the script issues error code L006 indicating the specific missing fields.

Root Structure Validation

The validator accepts two top-level structures (lines 39–45). The file must contain either a non-empty array of question objects directly, or a JSON object containing a non-empty questions array. Any other structure triggers a validation failure before individual question inspection begins.

Question-Level Validation Rules

Once parsed, every entry in the questions array must pass type and content constraints implemented in scripts/audit_lessons.py.

Object Type Verification

Each question must be a JSON object. The script explicitly checks isinstance(q, dict) (lines 53–56), issuing error L006 if an array entry is not a dictionary.

Option Count Constraints

The options field must be an array containing 2 to 6 items. The validation logic (lines 76–84) verifies both that options is a list and that 2 <= len(options) <= 6. Failure results in error code L008.

Correct Index Verification

The correct field must be an integer index pointing to a valid position within the options array. The validator confirms isinstance(correct, int) and checks the bounds 0 <= correct < len(options) (lines 86–93). An out-of-bounds index or non-integer value triggers error L009.

Stage Structure Conventions

While the audit scripts do not programmatically enforce the exact distribution of stages, the curriculum documentation in AGENTS.md specifies a six-question structure: one pre stage, three check stages, and two post stages. The stage key in each question object must contain one of these string values to maintain consistency across lesson files.

Summary

• Automated enforcement: The CI pipeline runs scripts/audit_lessons.py on every PR to validate quiz.json files.
• Strict key schema: Questions must use the canonical keys (stage, question, options, correct, explanation) and cannot contain legacy aliases (q, choices, answer).
• Structural rules: Files must contain a non-empty array of questions or an object with a questions array.
• Content constraints: Questions require 2–6 options, and the correct field must be a valid 0-based integer index within those options.
• Error codes: Validation failures return specific codes (L006, L007, L008, L009) to guide authors toward precise fixes.

Frequently Asked Questions

What happens if I use legacy keys like "q" or "answer" in my quiz.json?

The validator detects legacy keys by checking for intersection with LEGACY_QUIZ_KEYS and immediately issues error code L007. The build fails, and the PR cannot merge until you replace legacy keys with their canonical equivalents (question for q, options for choices, correct for answer).

How many options must each quiz question have?

Every question must contain between 2 and 6 options inclusive. The audit script validates the length of the options array (lines 76–84), and any value outside this range produces error L008.

What is the valid range for the "correct" index?

The correct value must be a zero-based integer index that falls within the bounds of the options array. For an array of length n, valid values range from 0 to n-1. The validator checks this constraint at lines 86–93 and reports violations as error L009.

Does the validation enforce the six-question stage structure?

The automated scripts validate that the stage key exists and is a string, but they do not enforce the specific distribution of one pre, three check, and two post stages. This six-question convention is documented in AGENTS.md and expected to be followed manually by curriculum authors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}