How Lessons Are Structured in AI Engineering From Scratch: A Complete Guide to code/, docs/en.md, outputs/, and quiz.json

Every lesson in the AI Engineering From Scratch repository follows a strict four-part structure containing documentation (docs/en.md), executable code (code/), reusable output artifacts (outputs/), and a standardized assessment (quiz.json) to create self-contained learning packages.

The AI Engineering From Scratch curriculum organizes hands-on machine learning education into reproducible, automation-friendly lesson modules. Each lesson follows a standardized directory structure that enables automated validation, site generation, and skill reuse across the repository.

The Four-Part Lesson Architecture

The repository enforces a rigid layout through CI scripts like scripts/audit_lessons.py. Every lesson directory must contain four specific components that work together to provide documentation, implementation, reusable assets, and assessment.

Documentation (docs/en.md)

The docs/en.md file serves as the human-readable lesson guide containing learning objectives, conceptual background, and step-by-step instructions. This file must start with YAML frontmatter defined in AGENTS.md that specifies the type, language, prerequisites, and time estimate for the lesson.

The language field in the frontmatter must match the file extensions found in the lesson's code/ directory. For example, a lesson marked with language: Python should contain code/main.py or similar Python files.

Code Implementation (code/)

The code/ directory contains the runnable implementation that learners build and test. Files follow the convention code/main.<ext> where the extension matches the documented language. Each file includes a header comment citing the path to the lesson's docs/en.md file and any external specifications.

Code implementations are self-terminating scripts without long-running stdin loops. Tests reside in code/tests/ and execute via standard language-specific runners like python -m unittest discover.

Output Artifacts (outputs/)

The outputs/ directory stores reusable assets generated by the lesson, such as skill prompts, reference diagrams, or compiled model checkpoints. While file names follow informal conventions like skill-<slug>.md or prompt-<slug>.md, all artifacts must reside under this directory.

Other lessons can import these outputs as dependencies, making the curriculum modular and composable.

Assessment (quiz.json)

The quiz.json file contains a six-question assessment that conforms to the schema defined in AGENTS.md. The structure follows a specific pattern: one pre question, three check questions, and two post questions.

Each question object includes stage, question, options, correct (zero-based index), and explanation fields. The curriculum site uses this file to render interactive quizzes and validate learner comprehension.

Directory Structure and Naming Conventions

Lessons live under a hierarchical path structure: phases/<phase-number>-<phase-slug>/<lesson-number>-<lesson-slug>/. For example, the tokenizers lesson resides at phases/10-llms-from-scratch/01-tokenizers/.

The four required subdirectories (docs, code, outputs) and the root file quiz.json are mandatory. The site generator (site/build.js) automatically discovers lessons by scanning for these specific paths, extracting titles and links without manual configuration.

Metadata Flow and CI Integration

The repository's automation relies on the strict consistency of the AI Engineering From Scratch lesson structure. The scripts/audit_lessons.py CI script validates that every lesson contains the required files and that frontmatter languages match code file extensions.

When the CI pipeline runs, it reads the frontmatter from docs/en.md to populate README.md tables and ROADMAP.md status indicators. This metadata flow ensures that curriculum documentation stays synchronized with the actual lesson content.

Working with Lesson Files Programmatically

You can interact with lesson assets programmatically using Python to automate curriculum validation or build custom tooling:

import json
from pathlib import Path

# Base path to the lesson (change the lesson slug as needed)

lesson_root = Path("phases/10-llms-from-scratch/01-tokenizers")

# 1. Load the markdown documentation

doc_path = lesson_root / "docs" / "en.md"
doc_text = doc_path.read_text(encoding="utf-8")
print("Lesson title:", doc_text.splitlines()[0].lstrip('# ').strip())

# 2. Import and run the lesson code

code_path = lesson_root / "code" / "main.py"
exec(code_path.read_text(), {"__name__": "__main__"})   # runs the demo script

# 3. Access an output artifact

output_path = lesson_root / "outputs" / "skill-tokenizer.md"
print("\n--- Output Artifact ---")
print(output_path.read_text())

# 4. Parse the quiz definition

quiz_path = lesson_root / "quiz.json"
quiz = json.loads(quiz_path.read_text())
print("\nQuiz questions:")
for q in quiz["questions"]:
    print(f'- ({q["stage"]}) {q["question"]}')

Summary

• Four required components: Every lesson must contain docs/en.md, code/, outputs/, and quiz.json to pass CI validation.
• Strict path conventions: Lessons follow the pattern phases/NN-phase-slug/NN-lesson-slug/ with fixed subdirectory names.
• Metadata-driven automation: Frontmatter in docs/en.md drives the site generator (site/build.js) and CI scripts (scripts/audit_lessons.py).
• Standardized assessment: quiz.json follows a six-question schema with specific stages (pre, check, post) and zero-based correct answer indices.
• Reusable outputs: The outputs/ directory enables skill sharing between lessons through informal naming conventions like skill-<slug>.md.

Frequently Asked Questions

What happens if a lesson is missing one of the four required components?

The CI pipeline will fail. The scripts/audit_lessons.py script validates that every lesson directory contains docs/en.md, the code/ and outputs/ subdirectories, and quiz.json. Missing components prevent the site generator from building the lesson page and block merges to the main branch.

How does the quiz.json schema work in AI Engineering From Scratch?

The quiz.json file must contain exactly six questions following the schema defined in AGENTS.md. The structure requires one pre question (pre-assessment), three check questions (mid-lesson verification), and two post questions (final assessment). Each question includes an options array, a correct field with a zero-based index indicating the right answer, and an explanation field for feedback.

Can lessons depend on outputs from other lessons?

Yes. The outputs/ directory is designed for reusable artifacts that other lessons can import. Files like skill-tokenizer.md or prompt-<slug>.md generated in one lesson can be referenced by subsequent lessons in the curriculum, creating a modular learning path where skills build upon previous outputs.

Why must code files include a header comment citing docs/en.md?

The header comment creates an explicit link between the implementation and its documentation, enabling automated tooling to verify that code files match their described lesson plans. This requirement ensures that learners can always trace executable code back to its conceptual explanation in docs/en.md, and helps maintain curriculum integrity when paths or structures change.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}