Assessment Schema for Diagnostics: Handling Single vs. Multiple Response Questions

The Claude certification assessment schema for diagnostics employs a type field set to either "single" or "multiple" to distinguish question formats, with the correct field always structured as a zero-based index array to maintain consistent validation logic across the test harness.

The rohitg00/ai-engineering-from-scratch repository implements a unified assessment schema for diagnostics that accommodates both single-choice and multiple-choice questions within the same JSON structure. This design allows certification tracks to seamlessly mix question types while enforcing strict validation rules. The schema is formally defined in AGENTS.md and realized in concrete assessment files across the certifications/claude/assessments/ directory.

Core Structure of the Diagnostic Schema

The diagnostic schema consists of two hierarchical layers: the diagnostic wrapper containing metadata about the assessment as a whole, and the individual question objects defining specific test items.

Top-Level Diagnostic Metadata

Each diagnostic file begins with a top-level object that describes global assessment properties. According to the schema documentation in AGENTS.md, these fields include:

• id – Unique identifier for the diagnostic (e.g., claude-ccar-f-diagnostic)
• version – Schema version (currently 1)
• track – Certification track the diagnostic belongs to (e.g., ccar-f)
• kind – Assessment classification, always "diagnostic" for diagnostic files
• title – Human-readable assessment name
• timeLimitMinutes – Maximum time allowed for completion
• questions – Array of question objects containing the actual test content

Question-Level Properties

Within the questions array, each question object follows a standardized structure defined in the schema:

• id – Stable identifier (e.g., ccar-f-agent-001)
• domain – High-level knowledge area being tested (e.g., agentic-architecture-orchestration)
• objective – Specific learning objective (e.g., choose-an-orchestration-pattern)
• type – Question format discriminator: "single" for single-response or "multiple" for multiple-response
• prompt – The question stem presented to the user
• options – Array of answer choices (typically labeled "a", "b", "c", "d")
• correct – Array of zero-based integers indicating correct option indices
• explanation – Narrative justifying the correct answer(s)
• references (optional) – Array of paths to lesson files providing background material

Configuring Single vs. Multiple Response Questions

The schema distinguishes between question types through the type field while maintaining a uniform structure for the correct field across both variants.

Single-Response Question Format

For questions expecting exactly one correct answer, set the type field to "single". In this configuration, the correct field contains a single-element array with the zero-based index of the correct option.

The following excerpt from certifications/claude/assessments/ccar-f/diagnostic.json demonstrates a single-response question:

{
  "id": "claude-ccar-f-diagnostic",
  "version": 1,
  "track": "ccar-f",
  "kind": "diagnostic",
  "title": "Architect Foundations Diagnostic",
  "timeLimitMinutes": 30,
  "questions": [
    {
      "id": "ccar-f-agent-001",
      "domain": "agentic-architecture-orchestration",
      "objective": "choose-an-orchestration-pattern",
      "type": "single",
      "prompt": "A self-contained original scenario...",
      "options": ["a", "b", "c", "d"],
      "correct": [1],
      "explanation": "Why the decision fits and the alternatives do not.",
      "references": ["certifications/claude/lessons/16-multi-agent-orchestration-and-delegation"]
    }
  ]
}

Multiple-Response Question Format

For questions allowing several correct selections, set the type field to "multiple". The correct field then contains an array of all valid zero-based indices.

The following example from certifications/claude/assessments/ccdv-f/mock-01.json illustrates the multiple-response pattern:

{
  "id": "ccdv-f-mock-01",
  "version": 1,
  "track": "ccdv-f",
  "kind": "mock",
  "title": "Developer Foundations Mock",
  "timeLimitMinutes": 30,
  "questions": [
    {
      "id": "ccdv-f-agent-007",
      "domain": "model-deployment",
      "objective": "select-necessary-steps",
      "type": "multiple",
      "prompt": "Which of the following steps are required to safely deploy a model?",
      "options": ["a", "b", "c", "d", "e"],
      "correct": [0, 2, 4],
      "explanation": "Steps a, c, and e cover vetting, monitoring, and rollback, which are essential for secure deployment."
    }
  ]
}

Validation Logic and the Type Field

The type field directly drives validation behavior in the test harness. When processing questions from files like certifications/claude/assessments/ccar-p/diagnostic.json, the system applies the following rules:

• Single-response validation: Expects the correct array to contain exactly one integer (or validates a single integer value, depending on implementation) representing the sole correct answer index
• Multiple-response validation: Expects the correct array to contain one or more integers, with all listed indices considered valid answers

Both formats use zero-based indexing for the correct field, aligning with standard programming conventions and simplifying tooling development.

Key Implementation Files

The assessment schema for diagnostics is implemented across several critical files in the repository:

• AGENTS.md – Contains the canonical schema documentation and field descriptions
• certifications/claude/assessments/ccar-f/diagnostic.json – Real diagnostic file demonstrating mixed single and multiple question types
• certifications/claude/assessments/ccdv-f/mock-01.json – Mock assessment featuring multiple-response questions
• certifications/claude/assessments/ccar-p/diagnostic.json – Additional diagnostic example for a different certification track

Summary

• The type field distinguishes single-response ("single") from multiple-response ("multiple") questions in the assessment schema
• The correct field is always structured as an array of zero-based integers, even for single-choice questions, ensuring uniform data shapes across the schema
• Question objects include metadata fields (domain, objective) for learning analytics, alongside presentation fields (prompt, options)
• Reference links connect diagnostic questions to specific lesson files, creating traceable learning pathways
• Validation rules are determined by the type field, with the test harness enforcing appropriate single vs. multiple selection constraints

Frequently Asked Questions

What determines if a question accepts single or multiple responses?

The type field in the question object determines the response format. Set "type": "single" for questions with exactly one correct answer, or "type": "multiple" for questions where several options may be correct. The test harness uses this field to apply appropriate validation logic when scoring responses.

Why does the correct field use an array for single-response questions?

The schema enforces a uniform array structure for the correct field across all question types to simplify tooling and validation code. For single-response questions, the array contains exactly one integer (e.g., [1]), while multiple-response questions contain multiple integers (e.g., [0, 2, 4]). This consistency eliminates type-checking complexity in the test harness implementation.

How are correct answer indices represented in the schema?

All indices in the correct field use zero-based numbering, where 0 represents the first option in the options array, 1 represents the second, and so forth. This aligns with standard programming conventions used throughout the repository's tooling infrastructure.

Where is the official assessment schema documentation maintained?

The authoritative schema documentation resides in AGENTS.md at the repository root. Concrete implementations demonstrating both single and multiple response patterns appear in track-specific files such as certifications/claude/assessments/ccar-f/diagnostic.json and certifications/claude/assessments/ccdv-f/mock-01.json.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}