How Reusable Artifacts (Prompts, Skills, Agents, and MCP Servers) Are Generated for Each Lesson in AI Engineering From Scratch

The curriculum generates reusable artifacts through an automated discovery pipeline that parses author-written markdown files with YAML front-matter, filters them by type and tags, and installs them into configurable layouts while generating a version-controlled manifest.

The rohitg00/ai-engineering-from-scratch repository treats prompts, skills, agents, and MCP servers as first-class reusable artifacts. Each lesson produces self-describing components stored in outputs/ directories, which downstream lessons and external tools can discover and install via a Python CLI. This system ensures that every prompt template, skill definition, and agent configuration created during the curriculum remains discoverable, versioned, and reusable.

The Artifact Generation Pipeline

The generation process follows a strict file-naming convention and automated discovery cycle. Instead of manual copying, the repository uses scripts/install_skills.py to walk the entire curriculum tree and extract structured metadata from markdown files.

Author-Written Artifact Files

Each lesson contains an outputs/ folder holding one or more markdown files that define reusable components. The filename follows the pattern {type}-{slug}.md, where type can be skill, prompt, agent, or skill-mcp-server (for MCP servers).

For example, the MCP server artifact for the capstone project resides at:

phases/19-capstone-projects/13-mcp-server-with-registry/outputs/skill-mcp-server.md

These files contain both the artifact content (markdown body) and the metadata required to catalog and install it.

YAML Front-Matter Metadata

At the top of every artifact markdown file, authors add a YAML block that describes the component. The scripts/_lib.py module provides parse_frontmatter() to extract this metadata during the discovery phase.

A typical skill front-matter looks like this:

---
name: MCP Server
type: skill
phase: 13
lesson: 13
version: "1.0"
description: Deploy a production MCP server with StreamableHTTP, OAuth scopes, and OPA policy.
tags:
  - mcp
  - server
---

This self-describing format ensures that artifacts carry their own versioning and categorization, making them independent of the directory structure.

Discovery and Parsing

The discover_artifacts() function in scripts/install_skills.py walks the phases/**/outputs tree to build an inventory of available components. For each .md file encountered, the script:

1. Extracts the artifact type from the filename prefix
2. Parses the YAML front-matter using parse_frontmatter from scripts/_lib.py
3. Falls back to directory numbers (phase-NN, lesson-MM) when metadata fields are missing
4. Instantiates an Artifact dataclass with fields for type, name, phase, lesson, version, description, tags, and source path

This discovery mechanism runs automatically during CI and can be invoked locally to inspect the curriculum's available assets.

Filtering and Installation

Users invoke install_skills.py with filters to select specific artifacts. The script provides filter_artifacts() to narrow results by --type, --phase, or --tag, and uses target_path() to determine the destination based on the chosen --layout strategy (flat, by-phase, or skill-folder).

The CLI supports the following options:

• --type skill – Filter for specific artifact types (skill, prompt, agent)
• --phase 13 – Limit to artifacts from a specific curriculum phase
• --tag mcp – Filter by tags defined in the YAML front-matter
• --layout by-phase – Organize output into phase-based directories
• --dry-run – Preview actions without writing files

Installing and Reusing Artifacts

The installation process copies selected artifacts to a target directory and generates a machine-readable manifest. This allows downstream lessons to import skills from previous phases or lets external tools consume the curriculum's prompts and agents.

Command Examples

Install every skill artifact from the entire curriculum into a local directory:

python3 scripts/install_skills.py ./my-skills/ --type skill --layout by-phase

Preview which agent artifacts would be installed without writing files:

python3 scripts/install_skills.py ./my-skills/ --type agent --dry-run

Install only MCP-related artifacts from Phase 13 using a flat layout:

python3 scripts/install_skills.py ./my-mcp/ \
    --type skill --phase 13 --tag mcp --layout flat

Manifest Generation

After copying files, write_manifest() generates a manifest.json in the target directory. This JSON file summarizes every installed artifact with its type, name, phase, lesson, source path, target path, tags, and version. The manifest serves as a contract between the curriculum source and consuming applications, ensuring exact reproducibility of which artifacts are available at any given version.

Site Generation and the Data Layer

The static site generator consumes the artifact metadata to render the "outputs" section of each lesson page. The site/build.js script reads the manifest (via site/data.js generated at CI time) to expose reusable artifacts to readers browsing the curriculum online.

This creates a bridge between the source code repository and the documentation website, ensuring that the artifact catalog visible to users matches the actual files available in the phases/**/outputs directories.

Summary

• Self-describing files: Artifacts use YAML front-matter in phases/**/outputs/{type}-{slug}.md to declare their metadata, version, and tags.
• Automated discovery: scripts/install_skills.py uses discover_artifacts() to parse all markdown files and build an inventory without manual registration.
• Flexible filtering: The CLI supports filtering by --type, --phase, and --tag to install only relevant components.
• Version-controlled manifests: write_manifest() generates a manifest.json that records exact source paths and versions for reproducibility.
• Multi-format output: The --layout option supports flat, by-phase, or skill-folder structures to suit different consumption patterns.
• Site integration: site/build.js renders the artifact catalog for web readers using the same manifest consumed by the CLI.

Frequently Asked Questions

How does the system distinguish between different artifact types?

The system extracts the artifact type from the filename prefix using the pattern {type}-{slug}.md. Valid types include skill, prompt, agent, and skill-mcp-server. This convention allows the discover_artifacts() function to categorize files without opening them, while the YAML front-matter inside each file provides additional metadata like specific tags and version numbers.

Can I install artifacts from multiple phases at once?

Yes. The install_skills.py script accepts any combination of filters. Omitting the --phase argument causes the discovery to walk the entire phases/**/outputs tree, collecting artifacts from all curriculum phases. You can combine this with --type or --tag to create cross-phase collections of specific components, such as all MCP-related skills across the entire curriculum.

What happens if the YAML front-matter is missing required fields?

The discover_artifacts() function implements fallback logic that derives metadata from the directory structure. If the YAML block lacks phase or lesson numbers, the script parses the parent directory names (e.g., phase-19/lesson-13) to populate the Artifact dataclass. However, fields like name, description, and version should be explicitly defined in the front-matter for reliable identification.

Where is the artifact manifest used outside of installation?

The manifest.json generated by write_manifest() serves multiple consumers. The site/build.js script reads it (via site/data.js) to populate the curriculum website's "outputs" sections, while external tools can parse it to programmatically discover available skills and prompts. This JSON file acts as the single source of truth for what reusable artifacts exist in the repository at any given commit.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}