Requirements for Certification Lessons in AI Engineering from Scratch: Complete Technical Checklist

Certification lessons in the AI Engineering from Scratch repository must satisfy a strict "full-parity" contract enforced by scripts/audit_certifications.py, including mandatory YAML front-matter, 800+ words of prose, five specific content sections, runnable code with unit tests, and a six-question quiz schema.

The rohitg00/ai-engineering-from-scratch curriculum enforces rigorous standards for certification lessons to ensure every module serves as a complete, runnable artifact. These requirements, documented in AGENTS.md and validated by automated auditing scripts, create a consistent structure across all certifications/claude/lessons/ entries. Meeting the full-parity certification contract guarantees that lessons provide interactive labs, verifiable outputs, and assessment-ready quizzes.

Folder Structure and Layout Requirements

Every certification lesson must reside in its own directory under certifications/claude/lessons/<slug>/. According to the repository layout defined in AGENTS.md, each lesson folder must contain four specific subdirectories and files:

• docs/ – containing the primary en.md file with lesson content
• code/ – containing executable source files including main.<ext>
• outputs/ – containing at least one reusable artifact (skill, prompt, agent, or MCP server)
• quiz.json – containing the assessment questions following the strict schema

The audit script at scripts/audit_certifications.py (lines 100-104) explicitly checks for the presence of output artifacts in the outputs/ directory, flagging any lesson that fails to produce a tangible deliverable.

Content and Formatting Standards

Mandatory Front-Matter

The docs/en.md file must begin with a YAML-style front-matter block. As specified in AGENTS.md (lines 70-80), the block must include four mandatory fields: Type, Languages, Prerequisites, and Time. The audit script (lines 106-108) validates that these fields exist and are non-empty.

Minimum Prose Length and Structure

The lesson text must contain at least 800 words; otherwise, audit_certifications.py (line 85) flags the lesson as "too thin for certification preparation". Additionally, the file must include:

• An H1 heading (# Title) for indexing (validated at line 86)

• An H2 section titled "Learning Objectives" (validated at line 88)

The Five Full-Parity Sections

The PARITY_HEADINGS constant in audit_certifications.py (lines 67-73) defines five mandatory sections that every certification lesson must contain:

1. Interactive Lab – Hands-on exercises with embedded figures
2. Practice Lab – Additional exercises for skill reinforcement
3. Shipped Artifact – Deliverable code or configuration
4. Verify It – Steps to validate the implementation
5. Capstone Connection – Links to broader curriculum goals

These sections ensure pedagogical consistency across the AI Engineering from Scratch certification track.

Code and Testing Requirements

Runnable Main File

Every lesson must include a code/main.<ext> file that can be executed end-to-end. Supported languages include Python, TypeScript, Rust, and Julia. The Languages field in the front-matter must match the actual extension of the main file (validated in audit_certifications.py, lines 82-95). If no code is required, the lesson must explicitly declare none or n/a.

Header Citation

The top comment of the main file must reference the lesson's docs/en.md path. The audit script (lines 109-111) checks for this citation to maintain traceability between documentation and implementation.

Minimum Test Count

For Python lessons, the code/tests/ directory must contain at least 5 test functions (def test_...). Other languages follow the same five-test minimum, as enforced by the audit logic at lines 112-118.

Quiz Schema and Assessment Structure

The quiz.json file must adhere to a strict schema defined in AGENTS.md (lines 88-101). The requirements include:

• Exactly six questions total: 1 pre-assessment, 3 check-for-understanding, and 2 post-assessment
• Each question must include stage, question, options (array of 4 strings), correct (zero-based index), and explanation (minimum 20 characters)

The audit_certifications.py script (lines 68-70) validates the stage distribution, ensuring the 1-3-2 ratio is maintained across all lessons.

Figure Registration and Assets

For lessons with slugs beginning with a two-digit prefix (e.g., 01-...), the EXPECTED_FIGURES mapping (lines 74-90 in audit_certifications.py) requires a matching figure ID to appear in a fenced code block within the lesson. This figure ID must also be registered in site/figures/runtime.js (validated at lines 95-100).

Validation and Auditing

The scripts/audit_certifications.py script serves as the automated validator for the full-parity certification contract described in AGENTS.md (lines 107-129). This script performs comprehensive checks including:

• Word count validation
• Required heading verification
• Front-matter field presence
• Code language consistency
• Test count verification
• Quiz schema compliance
• Output artifact existence

A lesson that passes all audit checks without errors is considered fully compliant with the AI Engineering from Scratch certification standards.

Code Examples

Below are minimal implementations that satisfy the most common certification requirements.

docs/en.md Front-Matter

# Title of the Certification Lesson

> One-line hook that captures the lesson's purpose.

**Type:** Build  
**Languages:** Python  
**Prerequisites:** None  
**Time:** ~120 minutes

## Learning Objectives

- Understand core concepts
- Build runnable implementations
- Validate against test suites

## Interactive Lab

...

## Practice Lab

...

## Shipped Artifact

...

## Verify It

...

## Capstone Connection

...

Embedded Figure Block

For lessons requiring figures (e.g., slug 01-claude-product-and-model-landscape):

```figure
01-claude-model-fit

```

code/main.py with Header Citation

# docs/en.md: certifications/claude/lessons/01-claude-product-and-model-landscape/docs/en.md

def main():
    """Entry point for the certification lesson."""
    print("Hello, certification!")
    return 0

if __name__ == "__main__":
    main()

Minimum Test Suite (code/tests/test_main.py)

def test_main_runs():
    """Verify main function executes without error."""
    assert True

def test_core_logic():
    """Test primary business logic."""
    assert 1 + 1 == 2

def test_edge_case_empty():
    """Test edge case with empty input."""
    assert len("") == 0

def test_string_manipulation():
    """Test string operations."""
    assert "a".upper() == "A"

def test_type_validation():
    """Verify type checking works."""
    assert isinstance([], list)

quiz.json Schema

{
  "lesson": "01-claude-product-and-model-landscape",
  "title": "Claude Product and Model Landscape",
  "questions": [
    {
      "stage": "pre",
      "question": "What is the primary purpose of the Claude API?",
      "options": ["Text generation", "Image editing", "Database management", "Network routing"],
      "correct": 0,
      "explanation": "Claude is primarily designed for text generation and conversational AI tasks."
    },
    {
      "stage": "check",
      "question": "Which parameter controls maximum output length?",
      "options": ["temperature", "max_tokens", "top_p", "frequency_penalty"],
      "correct": 1,
      "explanation": "The max_tokens parameter explicitly limits the number of tokens in the response."
    },
    {
      "stage": "check",
      "question": "Which SDK is officially supported?",
      "options": ["claude-python", "anthropic", "claude-ai", "anthropic-sdk"],
      "correct": 1,
      "explanation": "The official SDK is available as the anthropic package on PyPI."
    },
    {
      "stage": "check",
      "question": "What format does the API accept?",
      "options": ["XML", "YAML", "JSON", "CSV"],
      "correct": 2,
      "explanation": "The Claude API accepts and returns JSON-formatted requests and responses."
    },
    {
      "stage": "post",
      "question": "When should you use system prompts?",
      "options": ["Never", "Only for chat", "For setting context and instructions", "Only in testing"],
      "correct": 2,
      "explanation": "System prompts are used to set context, provide instructions, and guide model behavior."
    },
    {
      "stage": "post",
      "question": "What is the benefit of streaming responses?",
      "options": ["Lower cost", "Faster perceived time", "Better accuracy", "More tokens"],
      "correct": 1,
      "explanation": "Streaming provides faster perceived response time by delivering tokens as they are generated."
    }
  ]
}

Summary

• Certification lessons must reside in certifications/claude/lessons/<slug>/ with docs/, code/, outputs/, and quiz.json
• docs/en.md requires YAML front-matter with Type, Languages, Prerequisites, and Time, plus 800+ words and an H1 heading
• The five full-parity sections (Interactive Lab, Practice Lab, Shipped Artifact, Verify It, Capstone Connection) are mandatory
• code/main.<ext> must be runnable with a header citation referencing the docs path
• Minimum 5 unit tests required in code/tests/
• quiz.json must contain exactly 6 questions (1 pre, 3 check, 2 post) with 20+ character explanations
• Lessons with numeric prefixes require registered figures in site/figures/runtime.js
• Automated validation occurs via scripts/audit_certifications.py against the contract in AGENTS.md

Frequently Asked Questions

What happens if my lesson fails the audit_certifications.py script?

The script will output specific error messages indicating which requirements are missing, such as "lesson is too thin for certification preparation" for word count violations or missing mandatory front-matter fields. You must address all flagged issues before the lesson can be merged into the certification track.

Can I use a language other than Python for the main file?

Yes, the repository supports Python, TypeScript, Rust, and Julia. The Languages field in your front-matter must match the extension of your code/main.<ext> file. If you choose a language without a specific test runner configured, you must still provide at least 5 test functions following the same naming conventions.

Why is the 800-word minimum enforced?

The 800-word minimum ensures that certification lessons provide sufficient depth for learners to understand complex AI engineering concepts, complete hands-on labs, and connect theory to practice. This threshold prevents superficial content that would inadequately prepare students for the capstone assessments.

Are generated files like site/data.js allowed in my lesson folder?

No, AGENTS.md hard rule 7 explicitly prohibits committing generated files such as site/data.js or catalog.json. These files are regenerated by the CI pipeline and must never be included in your pull request. Only source materials (docs, code, tests, quiz.json, and output artifacts) should be committed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}