How Skill Installation Works in AI Engineering from Scratch: A Deep Dive into install_skills.py

The install_skills.py script in the rohitg00/ai-engineering-from-scratch repository is a command-line utility that extracts skill, prompt, and agent markdown artifacts from the curriculum's phases/**/outputs folders and copies them into a user-specified target directory through a four-stage pipeline of discovery, filtering, planning, and execution.

The skill installation process enables users to package and deploy reusable learning artifacts from the AI Engineering from Scratch curriculum. The scripts/install_skills.py utility automates this extraction, allowing selective installation of skills, prompts, and agents from specific phases while preserving metadata and versioning information.

The Four-Stage Skill Installation Pipeline

The script operates through a deterministic pipeline that transforms source markdown into organized, deployable artifacts.

Stage 1: Artifact Discovery

The discover_artifacts() function (lines 94-136 in scripts/install_skills.py) walks every phases/*/outputs directory to locate markdown files. It filters for filenames starting with skill-, prompt-, or agent-, then reads the markdown content and parses YAML front-matter using _lib.parse_frontmatter. Each discovered file becomes an Artifact object containing the type, name, phase, lesson, version, description, tags, and source path.

Stage 2: Filtering and Selection

The filter_artifacts() function (lines 139-154) applies user-specified criteria to narrow the discovered set. The script accepts three filter flags:

• --type: Restrict to skill, prompt, agent, or all
• --phase: Numeric phase ID (e.g., 7)
• --tag: Specific tag string for topical filtering

Only artifacts satisfying all active filters proceed to the planning stage.

Stage 3: Installation Planning

The build_plan() function (lines 168-191) maps filtered artifacts to destination paths based on the chosen layout strategy:

• flat: All files placed in the root directory
• by-phase: Organized into phase subdirectories
• skills: Default hierarchical structure

This stage detects path collisions (two artifacts targeting the same file) and records them as errors unless --force is enabled. The output is a list of (artifact, destination) tuples plus any collision paths.

Stage 4: Execution and Manifest Generation

The final stage differentiates between simulation and execution.

If --dry-run is specified, the script reports the planned operations without writing files.

Otherwise, apply_plan() (lines 194-226) creates parent directories as needed and copies each source markdown to its computed destination. Subsequently, write_manifest() (lines 229-267) generates a manifest.json in the target root recording every installed artifact with its computed target path, type, phase, tags, and aggregated statistics by type and phase.

Command-Line Interface and Usage

The main() function (lines 288-292) provides an argparse interface that orchestrates the pipeline and exits with status 0 on success or 1 on errors (such as no matching artifacts or unresolved collisions).

Basic installation with default settings:

python3 scripts/install_skills.py /path/to/target

Install all prompts for phase 7 using flat layout with overwrite enabled:

python3 scripts/install_skills.py /tmp/out --type prompt --phase 7 --layout flat --force

Preview agent artifacts without writing files:

python3 scripts/install_skills.py ./tmp --type agent --dry-run

Generate JSON manifest only for core-tagged artifacts:

python3 scripts/install_skills.py ./out --tag core --json

Key Implementation Files

The skill installation system relies on two primary modules:

• scripts/install_skills.py: Main driver implementing discovery, filtering, planning, and execution logic
• scripts/_lib.py: Helper module providing parse_frontmatter() for YAML metadata extraction

Source artifacts reside in phases/**/outputs/*.md as markdown files with structured front-matter.

Summary

• The skill installation process uses a four-stage pipeline: discovery, filtering, planning, and execution with manifest generation.
• The discover_artifacts() function parses YAML front-matter from phases/*/outputs directories to create typed Artifact objects.
• Filtering supports --type, --phase, and --tag flags to selectively install curriculum components.
• Layout strategies (flat, by-phase, skills) determine directory structure, with collision detection preventing overwrites unless --force is used.
• The manifest.json file provides a complete audit trail of installed artifacts with statistical aggregations.

Frequently Asked Questions

What file types does the install_skills.py script process?

The script processes markdown files (.md) located in phases/**/outputs directories whose names start with skill-, prompt-, or agent-. It extracts YAML front-matter from these files using parse_frontmatter() from _lib.py to build Artifact objects containing metadata such as type, phase, lesson, version, and tags.

How does the script handle file naming collisions?

During the planning stage, build_plan() detects when two artifacts map to the same destination path. Unless the --force flag is specified, these collisions are recorded as errors that halt installation with exit code 1. With --force, the script overwrites existing files with the latest planned artifact.

Can I preview changes before installing skills?

Yes. The --dry-run flag executes the discovery, filtering, and planning stages without calling apply_plan(). This displays exactly which files would be copied to which destinations and any potential collisions, allowing you to validate the operation before modifying the target directory.

What information does the manifest.json file contain?

The manifest.json file, generated by write_manifest() in the target root directory, records every installed artifact including its target path, type, phase, tags, and metadata. It also aggregates statistics by artifact type and phase, providing a comprehensive audit trail of the installation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}