# How the AI Engineering Skill Installation System Works: Inside scripts/install_skills.py

> Discover how the AI Engineering skill installation system operates via scripts/install_skills.py. Learn about its deterministic pipeline for copying curriculum artifacts and generating a JSON manifest. Explore the code now.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-07-30

---

**[`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py) implements a deterministic discover-filter-plan-apply pipeline that copies curriculum artifacts from the `phases` folder to a target directory while generating a JSON manifest of all installed items.**

The `ai-engineering-from-scratch` repository provides a structured curriculum for learning AI engineering through progressive phases. Central to its content distribution mechanism is [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py), a command-line utility that manages the extraction and installation of **skills**, **prompts**, and **agents** from the repository's structured phase directories into your local workspace. This script transforms scattered curriculum outputs into organized, reusable components through a systematic nine-stage process.

## The Installation Pipeline Architecture

The skill installation system follows a strict **discover → filter → plan → (dry-run) → apply → manifest** workflow. Each stage is implemented as a discrete function in [`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py), ensuring reproducible and configurable deployments.

### CLI Argument Parsing

At lines 30-36, the script uses Python’s `argparse` module to construct a flexible command-line interface. The parser accepts a positional `target` directory and optional filters including `--type`, `--phase`, `--tag`, `--layout`, `--dry-run`, `--force`, and `--json`. By default, the system installs only **skill** artifacts using the **skills** layout, making the simplest invocation `python3 scripts/install_skills.py ./target-dir`.

### Artifact Discovery

The `discover_artifacts()` function (lines 91-136) recursively walks every `phases/**/outputs` directory to locate markdown files prefixed with `skill-`, `prompt-`, or `agent-`. For each discovered file, it invokes `parse_frontmatter()` from [`_lib.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/_lib.py) to extract YAML metadata. When frontmatter is missing, the function derives phase and lesson numbers directly from the directory path structure.

### Content Filtering and Target Resolution

Discovered artifacts pass through `filter_artifacts()` (lines 39-54), which applies CLI-specified criteria (`type`, `phase`, `tag`) to exclude non-matching items. Simultaneously, `target_path()` (lines 57-65) computes destination paths based on the selected `--layout`:

- **`flat`**: Places files directly in `<target>/<name>.md`
- **`by-phase`**: Organizes files into `<target>/phase-NN/<name>.md` subdirectories
- **`skills`**: Creates structured directories at `<target>/<name>/SKILL.md`

### Installation Planning and Conflict Detection

The `build_plan()` function (lines 74-91) assembles a list of `(Artifact, destination)` tuples representing the complete installation strategy. This stage detects naming collisions—instances where multiple artifacts would write to the same destination—and emits warnings to `stderr` unless `--force` is specified to permit overwrites.

### Dry-Run Validation

When invoked with `--dry-run` (lines 64-77), the script prints a human-readable summary of planned actions and exits before modifying the filesystem. This mode respects the `--json` flag for machine-readable output, allowing CI/CD pipelines to validate installation plans programmatically.

### File System Execution

The `apply_plan()` function (lines 94-98) creates necessary parent directories using `os.makedirs` and copies source files to their computed destinations via `shutil.copy2`, preserving metadata such as timestamps and permissions.

### Manifest Generation

After successful copying, `write_manifest()` (lines 100-124) generates a [`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json) file in the target root. This JSON inventory records every installed artifact’s type, name, phase, lesson, source path, target path, tags, and version, along with summary statistics under a `totals` key. This manifest enables deterministic tracking of curriculum versions across distributed environments.

## Practical Usage Examples

The following commands demonstrate common installation scenarios supported by the system:

```bash

# Install all skills using the default "skills" layout

python3 scripts/install_skills.py ./my-skills

# Install only prompts from phase 12 with flat directory structure

python3 scripts/install_skills.py ./my-prompts \
  --type prompt --phase 12 --layout flat

# Preview changes without writing to disk

python3 scripts/install_skills.py ./tmp --dry-run

# Force overwrite existing files and filter by tag

python3 scripts/install_skills.py ./target \
  --tag beta --force

```

## Core Dependencies and Source Files

The installation system spans these critical components:

- **[`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py)**: Contains the main pipeline implementation, including argument parsing, filtering logic, and file operations.
- **[`scripts/_lib.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/_lib.py)**: Provides the `parse_frontmatter()` utility that extracts YAML metadata from curriculum markdown files.
- **`phases/**/outputs/*-{skill,prompt,agent}.md`**: Source artifacts distributed throughout the repository's phase structure that serve as installation sources.
- **[`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json)** (generated): The output inventory file written to the target directory containing complete installation metadata.

## Summary

- **[`scripts/install_skills.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/install_skills.py)** operates through a strict **discover-filter-plan-apply** pipeline that ensures reproducible curriculum installation.
- The system supports three layout modes (**flat**, **by-phase**, **skills**) to accommodate different organizational preferences.
- **Dry-run mode** (`--dry-run`) enables safe previewing of file system changes before execution.
- Every installation generates a **machine-readable manifest** ([`manifest.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/manifest.json)) that inventories installed artifacts with full provenance metadata.
- Collision detection prevents accidental overwrites unless explicitly overridden with `--force`.

## Frequently Asked Questions

### What artifact types can the installation system process?

The system recognizes three curriculum artifact types: **skills** (files prefixed with `skill-`), **prompts** (prefixed with `prompt-`), and **agents** (prefixed with `agent-`). All artifacts must be markdown files located within `phases/**/outputs` directories. You can filter by type using the `--type` command-line argument.

### How does the script handle file naming collisions?

During the planning phase (`build_plan()` at lines 74-91), the system detects when multiple artifacts would resolve to the same destination path. By default, it warns about collisions via `stderr` and aborts the installation. To overwrite existing files, you must explicitly pass the `--force` flag, which bypasses collision checks and permits destructive updates.

### Can I preview changes before modifying my file system?

Yes. The `--dry-run` flag executes the entire pipeline through the planning stage without calling `apply_plan()`. This mode outputs a summary of proposed copy operations and, when combined with `--json`, produces machine-readable output suitable for automation workflows. This short-circuit logic is implemented between lines 64-77 of the script.

### What information does the generated manifest.json contain?

The manifest records every installed artifact’s metadata including `type`, `name`, `phase`, `lesson`, `source` path, `target` path, `tags`, and `version`. It also includes a `totals` object with summary statistics. This file serves as a definitive record for auditing curriculum versions and tracking which specific artifacts are deployed in a given environment.