How to Ship Reusable Artifacts from the Outputs Directory in AI Engineering From Scratch

The rohitg00/ai-engineering-from-scratch repository ships reusable artifacts—skills, prompts, agents, and MCP servers—as plain data files via lesson-specific outputs/ directories, using the scripts/install_skills.py installer to discover, validate, and distribute them into target layouts while generating a machine-readable manifest.

The repository treats every lesson as a self-contained unit capable of exporting reusable components. These artifacts reside in outputs/ folders under individual lesson paths and are imported by downstream lessons through a language-agnostic installer that treats them as portable data rather than executable code.

The Artifact Shipping Pipeline

The shipping process centers on scripts/install_skills.py, which orchestrates five distinct phases to move artifacts from source lessons to target directories safely and predictably.

Discovery with discover_artifacts()

The installer first scans the entire repository to locate all outputs/ directories nested under lesson folders. The discover_artifacts() function builds an Artifact record for each file found, capturing metadata necessary for downstream processing. As implemented in scripts/install_skills.py at line 157, this phase recursively identifies potential exports without executing any code within them.

Security Validation via validate_output_directory()

Before copying, the installer validates each discovered directory to prevent directory traversal attacks. The validate_output_directory() function ensures each outputs/ folder is a regular directory—not a symlink that could escape the repository boundary. This security check at line 300 guarantees that only repository-contained files enter the shipping pipeline.

Layout Planning with build_plan()

The build_plan() function (line 271) determines the final destination for each artifact based on the selected layout strategy. The installer supports three layout modes:

• flat: All artifacts placed in a single directory
• by-phase: Organized under phase-specific subdirectories
• skills: Grouped by artifact type

Installation Strategies

Artifacts ship through two distinct installation paths depending on their structure:

install_flat_artifact() (line 542) handles single-file artifacts like standalone Markdown prompts or JSON configurations. It performs a direct copy into the target layout.

install_bundle() (line 502) processes complex artifacts packaged as directories containing a primary SKILL.md file plus supporting resources. This method preserves the entire bundle structure, ensuring dependent files like shell scripts or data reports remain accessible.

Manifest Generation with write_manifest()

Finally, write_manifest() (line 587) generates outputs/index.json—a centralized catalog listing every exported artifact with its type, source path, and metadata. Downstream lessons and external tools query this manifest to discover available resources without scanning the entire repository.

Command-Line Interface for Shipping Artifacts

Invoke the shipping process via the module command:

python -m scripts.install_skills \
  --type skill \
  --layout flat \
  --phase 15 \
  /path/to/target

Parameters include:

• --type: Filter by artifact category (skill, prompt, agent, or all)
• --layout: Choose the directory structure (flat, by-phase, or skills)
• --phase: Optional numeric filter to include only specific curriculum phases

Real-World Artifact Shipping Examples

Shipping a Skill Bundle

Lesson phases/15-autonomous-systems/04-darwin-godel-machine exports skill-dgm-evaluator-firewall.md and dgm-eval-report.json in its outputs/ folder. Running:

python -m scripts.install_skills --type skill --layout flat ./imported

Produces:

./imported/skills/skill-dgm-evaluator-firewall.md
./imported/skills/dgm-eval-report.json

Shipping an MCP Server Definition

The capstone lesson phases/19-capstone-projects/13-mcp-server-with-registry ships outputs/skill-mcp-server.md. Using --layout by-phase installs it to:

/path/to/target/phases/19-capstone-projects/13-mcp-server-with-registry/outputs/skill-mcp-server.md

The bundled run.sh script in the same directory enables immediate server launch after installation.

Summary

• Artifacts reside as plain files in lesson-specific outputs/ directories within the rohitg00/ai-engineering-from-scratch repository.
• The scripts/install_skills.py installer handles discovery, security validation, layout planning, and copying via functions like discover_artifacts() and install_bundle().
• Three layout strategies (flat, by-phase, skills) accommodate different consumption patterns.
• The generated outputs/index.json manifest provides machine-readable metadata for automated tooling.
• Both single-file artifacts and complex bundles ship safely without executing foreign code.

Frequently Asked Questions

What file types can be shipped from the outputs directory?

The shipping pipeline accepts any plain data files, including Markdown (.md), JSON (.json), and shell scripts (.sh). The install_flat_artifact() and install_bundle() functions treat these as opaque data, ensuring language-agnostic portability across different lessons and tools.

How does the installer prevent security issues when shipping artifacts?

The validate_output_directory() function in scripts/install_skills.py explicitly checks that each outputs/ directory is a regular directory and not a symbolic link. This prevents directory traversal attacks where a malicious symlink might point outside the repository boundary.

Can I ship multiple artifact types in a single command?

Yes. Use --type all to include skills, prompts, agents, and MCP servers simultaneously. The build_plan() function processes each artifact type according to its specific directory structure while maintaining separation in the target layout.

Where does the manifest file get generated?

The write_manifest() function generates outputs/index.json in the target directory specified during installation. This JSON catalog contains arrays for each artifact type—skills, prompts, agents, and mcp-servers—with paths and metadata that downstream automation can consume.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}