How the AI Engineering Skill Installation System Works: Inside scripts/install_skills.py

scripts/install_skills.py implements a deterministic discover-filter-plan-apply pipeline that copies curriculum artifacts from the phases folder to a target directory while generating a JSON manifest of all installed items.

The ai-engineering-from-scratch repository provides a structured curriculum for learning AI engineering through progressive phases. Central to its content distribution mechanism is scripts/install_skills.py, a command-line utility that manages the extraction and installation of skills, prompts, and agents from the repository's structured phase directories into your local workspace. This script transforms scattered curriculum outputs into organized, reusable components through a systematic nine-stage process.

The Installation Pipeline Architecture

The skill installation system follows a strict discover → filter → plan → (dry-run) → apply → manifest workflow. Each stage is implemented as a discrete function in scripts/install_skills.py, ensuring reproducible and configurable deployments.

CLI Argument Parsing

At lines 30-36, the script uses Python’s argparse module to construct a flexible command-line interface. The parser accepts a positional target directory and optional filters including --type, --phase, --tag, --layout, --dry-run, --force, and --json. By default, the system installs only skill artifacts using the skills layout, making the simplest invocation python3 scripts/install_skills.py ./target-dir.

Artifact Discovery

The discover_artifacts() function (lines 91-136) recursively walks every phases/**/outputs directory to locate markdown files prefixed with skill-, prompt-, or agent-. For each discovered file, it invokes parse_frontmatter() from _lib.py to extract YAML metadata. When frontmatter is missing, the function derives phase and lesson numbers directly from the directory path structure.

Content Filtering and Target Resolution

Discovered artifacts pass through filter_artifacts() (lines 39-54), which applies CLI-specified criteria (type, phase, tag) to exclude non-matching items. Simultaneously, target_path() (lines 57-65) computes destination paths based on the selected --layout:

• flat: Places files directly in <target>/<name>.md
• by-phase: Organizes files into <target>/phase-NN/<name>.md subdirectories
• skills: Creates structured directories at <target>/<name>/SKILL.md

Installation Planning and Conflict Detection

The build_plan() function (lines 74-91) assembles a list of (Artifact, destination) tuples representing the complete installation strategy. This stage detects naming collisions—instances where multiple artifacts would write to the same destination—and emits warnings to stderr unless --force is specified to permit overwrites.

Dry-Run Validation

When invoked with --dry-run (lines 64-77), the script prints a human-readable summary of planned actions and exits before modifying the filesystem. This mode respects the --json flag for machine-readable output, allowing CI/CD pipelines to validate installation plans programmatically.

File System Execution

The apply_plan() function (lines 94-98) creates necessary parent directories using os.makedirs and copies source files to their computed destinations via shutil.copy2, preserving metadata such as timestamps and permissions.

Manifest Generation

After successful copying, write_manifest() (lines 100-124) generates a manifest.json file in the target root. This JSON inventory records every installed artifact’s type, name, phase, lesson, source path, target path, tags, and version, along with summary statistics under a totals key. This manifest enables deterministic tracking of curriculum versions across distributed environments.

Practical Usage Examples

The following commands demonstrate common installation scenarios supported by the system:

# Install all skills using the default "skills" layout

python3 scripts/install_skills.py ./my-skills

# Install only prompts from phase 12 with flat directory structure

python3 scripts/install_skills.py ./my-prompts \
  --type prompt --phase 12 --layout flat

# Preview changes without writing to disk

python3 scripts/install_skills.py ./tmp --dry-run

# Force overwrite existing files and filter by tag

python3 scripts/install_skills.py ./target \
  --tag beta --force

Core Dependencies and Source Files

The installation system spans these critical components:

• scripts/install_skills.py: Contains the main pipeline implementation, including argument parsing, filtering logic, and file operations.
• scripts/_lib.py: Provides the parse_frontmatter() utility that extracts YAML metadata from curriculum markdown files.
• phases/**/outputs/*-{skill,prompt,agent}.md: Source artifacts distributed throughout the repository's phase structure that serve as installation sources.
• manifest.json (generated): The output inventory file written to the target directory containing complete installation metadata.

Summary

• scripts/install_skills.py operates through a strict discover-filter-plan-apply pipeline that ensures reproducible curriculum installation.
• The system supports three layout modes (flat, by-phase, skills) to accommodate different organizational preferences.
• Dry-run mode (--dry-run) enables safe previewing of file system changes before execution.
• Every installation generates a machine-readable manifest (manifest.json) that inventories installed artifacts with full provenance metadata.
• Collision detection prevents accidental overwrites unless explicitly overridden with --force.

Frequently Asked Questions

What artifact types can the installation system process?

The system recognizes three curriculum artifact types: skills (files prefixed with skill-), prompts (prefixed with prompt-), and agents (prefixed with agent-). All artifacts must be markdown files located within phases/**/outputs directories. You can filter by type using the --type command-line argument.

How does the script handle file naming collisions?

During the planning phase (build_plan() at lines 74-91), the system detects when multiple artifacts would resolve to the same destination path. By default, it warns about collisions via stderr and aborts the installation. To overwrite existing files, you must explicitly pass the --force flag, which bypasses collision checks and permits destructive updates.

Can I preview changes before modifying my file system?

Yes. The --dry-run flag executes the entire pipeline through the planning stage without calling apply_plan(). This mode outputs a summary of proposed copy operations and, when combined with --json, produces machine-readable output suitable for automation workflows. This short-circuit logic is implemented between lines 64-77 of the script.

What information does the generated manifest.json contain?

The manifest records every installed artifact’s metadata including type, name, phase, lesson, source path, target path, tags, and version. It also includes a totals object with summary statistics. This file serves as a definitive record for auditing curriculum versions and tracking which specific artifacts are deployed in a given environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}