How Skill Scripts Are Copied Into the Workspace and Made Available to Agents in Harvey-Labs

Harvey-Labs copies skill scripts into an isolated per-run workspace using shutil.copytree and exposes them to agents via a mounted sandbox directory, enabling direct bash execution during evaluation runs.

The harveyai/harvey-labs framework equips LLM-driven agents with reusable utilities for processing complex file formats like Excel, Word, and PowerPoint. To ensure each evaluation run operates within a clean, reproducible environment, the system automatically copies skill scripts into the workspace before mounting that directory into the execution sandbox.

Skill Resolution and Workspace Initialization

Before any files are copied, the framework determines which skills are required for the current evaluation run. This logic resides in the main() function within [harness/run.py](https://github.com/harveyai/harvey-labs/blob/main/harness/run.py).

Determining the Skill List

The system checks for the optional --skills command-line argument. When omitted, it defaults to loading all skills defined in DEFAULT_SKILLS. When provided, only the specified comma-separated skills are loaded. This resolution occurs at lines 86-89 of harness/run.py, where the skill list is parsed and normalized before proceeding to workspace setup.

Creating the Per-Run Workspace

For complete isolation between runs, the framework generates a fresh workspace directory at results/<run-id>/workspace. This directory serves as the mount point for the sandbox and the destination for all copied skill scripts. The path construction happens at lines 82-85 of harness/run.py:

results_dir = Path("results") / args.run_id
workspace_dir = results_dir / "workspace"
workspace_dir.mkdir(parents=True, exist_ok=True)

This ensures that workspace_dir exists and is empty for every new evaluation.

The Mechanism for Copying Skill Scripts Into the Workspace

Once the workspace exists, the framework invokes setup_skill_scripts() to populate it with utility files from the requested skill packages.

The setup_skill_scripts Implementation

Located at lines 17-24 of harness/run.py, this function iterates over the resolved skill list and copies the entire scripts/ subdirectory from each skill package into the workspace. The implementation uses shutil.copytree with dirs_exist_ok=True to handle overwrites gracefully:

def setup_skill_scripts(workspace_dir: Path, skills: List[str]) -> None:
    skills_dir = workspace_dir / "skills"
    for skill in skills:
        src = Path("harness/skills") / skill / "scripts"
        dst = skills_dir / skill / "scripts"
        shutil.copytree(src, dst, dirs_exist_ok=True)

This operation places scripts at workspace/skills/<skill>/scripts/, maintaining the original directory structure from the source repository.

Source and Destination Paths

The source scripts live within the repository under harness/skills/<skill>/scripts/. For example:

• harness/skills/xlsx/scripts/validate.py
• harness/skills/docx/scripts/template_fill.py

After copying, these become available at:

• results/<run-id>/workspace/skills/xlsx/scripts/validate.py
• results/<run-id>/workspace/skills/docx/scripts/template_fill.py

How Agents Access Copied Skill Scripts

After the copy operation completes, the framework starts the sandbox environment via [sandbox/sandbox.py](https://github.com/harveyai/harvey-labs/blob/main/sandbox/sandbox.py). The sandbox mounts the workspace directory, making the copied skill scripts directly accessible to the agent as ordinary filesystem utilities.

Executing Scripts Inside the Sandbox

The agent invokes these scripts using standard bash commands executed through the sandbox's ToolExecutor. Because the workspace is mounted as the working directory, scripts are referenced by their relative path:

bash skills/xlsx/scripts/validate.py input.xlsx
bash skills/docx/scripts/template_fill.py template.md data.json

This design decouples the agent from the host filesystem while providing full access to the isolated skill utilities within the container.

Running Evaluations with Custom Skill Sets

You can control which skill scripts are copied into the workspace using the --skills flag.

Running with Default Skills

To automatically copy all default skill scripts:

python -m harvey_labs.harness.run \
  --model claude-sonnet-5 \
  --task corporate-ma/review-data-room-red-flag-review \
  --run-id myrun123

Running with Specific Skills

To copy only the xlsx and docx skill scripts:

python -m harvey_labs.harness.run \
  --model claude-sonnet-5 \
  --task corporate-ma/review-data-room-red-flag-review \
  --run-id myrun456 \
  --skills xlsx docx

In both cases, the agent can execute commands like:

bash skills/xlsx/scripts/validate.py data.xlsx

Summary

• Skill resolution occurs in harness/run.py (lines 86-89), defaulting to DEFAULT_SKILLS unless the --skills flag is provided.
• Workspace creation happens at results/<run-id>/workspace (lines 82-85 of harness/run.py), ensuring isolated execution environments.
• Script copying is handled by setup_skill_scripts() (lines 17-24) using shutil.copytree(..., dirs_exist_ok=True) to populate workspace/skills/<skill>/scripts/.
• Agent access is granted through sandbox mounting, allowing bash execution of scripts via relative paths like skills/xlsx/scripts/validate.py.

Frequently Asked Questions

Where are skill scripts copied within the workspace?

Skill scripts are copied to results/<run-id>/workspace/skills/<skill>/scripts/. This structure mirrors the source location in harness/skills/<skill>/scripts/ and is created automatically by the setup_skill_scripts() function in harness/run.py.

How does the agent execute copied skill scripts?

The agent executes scripts by invoking bash commands with relative paths from the workspace root, such as bash skills/xlsx/scripts/validate.py. The sandbox mounts the workspace directory, so these commands execute inside the container with full access to the copied utilities.

What happens if script directories already exist in the workspace?

The setup_skill_scripts() function uses shutil.copytree(..., dirs_exist_ok=True), which overwrites existing files without raising errors. This ensures that each run starts with fresh copies of the skill scripts even if the workspace directory already contains data from previous operations.

Can I specify which skills to copy when running an evaluation?

Yes. Pass the --skills flag followed by space-separated skill names to the run command. If omitted, the system uses DEFAULT_SKILLS to copy all available skill scripts. This logic is implemented in the argument parsing section of main() in harness/run.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}