How AI-Scientist-v2 Ensures Experiment Workspace Isolation: A Complete Technical Guide

The AI-Scientist-v2 framework guarantees experiment workspace isolation by generating unique directory hierarchies for each run and creating process-specific subfolders for parallel workers, ensuring no file system collisions occur between concurrent experiments.

The SakanaAI/AI-Scientist-v2 repository implements a robust sandboxing architecture that prevents data contamination between autonomous research runs. Understanding how this system achieves experiment workspace isolation is critical for researchers running parallel agent workflows or managing large-scale automated experiments. The framework combines deterministic directory naming with process-aware file system operations to create fully isolated execution environments.

Core Workspace Isolation Architecture

The system implements a three-tier isolation strategy that operates at the experiment level, directory level, and process level.

Top-Level Workspace Creation with Unique Naming

In ai_scientist/treesearch/utils/config.py, the prep_cfg function establishes the root isolation layer. It resolves the user-supplied workspace_dir to an absolute path, calculates the next available experiment index, and appends a unique slug to prevent collisions.

# ai_scientist/treesearch/utils/config.py

top_workspace_dir = Path(cfg.workspace_dir).resolve()
top_workspace_dir.mkdir(parents=True, exist_ok=True)

# Generate unique experiment name using incremental index + random slug

ind = max(_get_next_logindex(top_log_dir), _get_next_logindex(top_workspace_dir))
cfg.exp_name = cfg.exp_name or coolname.generate_slug(3)
cfg.exp_name = f"{ind}-{cfg.exp_name}"

# Set concrete workspace path for this specific run

cfg.workspace_dir = (top_workspace_dir / cfg.exp_name).resolve()

This naming convention—combining an incremental integer with a randomly generated slug—ensures that even simultaneous launches with identical configuration files receive distinct directories.

Per-Experiment Subdirectory Sandboxing

Once the top-level directory is established, the prep_agent_workspace function creates isolated subdirectories for data and execution. According to the source code in ai_scientist/treesearch/utils/config.py, all file operations for a run are confined to these specific locations.

# Create isolated subdirectories for input data and working files

(cfg.workspace_dir / "input").mkdir(parents=True, exist_ok=True)
(cfg.workspace_dir / "working").mkdir(parents=True, exist_ok=True)

# Symlink or copy dataset into the sandboxed input folder

copytree(cfg.data_dir, cfg.workspace_dir / "input", use_symlinks=not cfg.copy_data)
if cfg.preprocess_data:
    preproc_data(cfg.workspace_dir / "input")

The input directory houses the experiment's dataset (either copied or symlinked), while the working directory contains all generated code, temporary files, and execution artifacts. This separation prevents experiments from modifying source data or leaking files into shared system directories.

Process-Level Isolation for Parallel Execution

When running in parallel mode, the system creates additional sandboxes within each experiment workspace. In ai_scientist/treesearch/parallel_agent.py, the _process_node_wrapper function constructs unique sub-folders for each multiprocessing worker using the process ID.

# ai_scientist/treesearch/parallel_agent.py

process_id = multiprocessing.current_process().name
workspace = os.path.join(cfg.workspace_dir, f"process_{process_id}")
os.makedirs(workspace, exist_ok=True)               # Isolated workspace per process

working_dir = os.path.join(workspace, "working")
os.makedirs(working_dir, exist_ok=True)            # Dedicated working directory

# Interpreter is confined to the process-specific path

process_interpreter = Interpreter(
    working_dir=workspace,
    timeout=cfg.exec.timeout,
    format_tb_ipython=cfg.exec.format_tb_ipython,
    agent_file_name=cfg.exec.agent_file_name,
)

This mechanism ensures that parallel agents operating on the same experiment node cannot overwrite each other's temporary files or create race conditions during file I/O operations.

Safety Mechanisms and Cleanup

Beyond directory isolation, the framework implements safeguards to maintain file system hygiene and prevent resource leakage.

Collision-Resistant Experiment Naming

The system employs a dual-naming strategy in prep_cfg that combines sequential indexing with entropy. The ind variable tracks the maximum existing directory index across both log and workspace directories, while coolname.generate_slug(3) adds a human-readable random suffix.

cfg.exp_name = f"{ind}-{cfg.exp_name}"

This approach guarantees that even if two experiments start within milliseconds of each other using the same base configuration, they receive unique folder names like 42-clever-penguin and 43-clever-penguin rather than colliding on identical strings.

Automatic Cleanup of Aborted Runs

To prevent empty directories from accumulating when experiments fail during initialization, ai_scientist/treesearch/perform_experiments_bfts_with_agentmanager.py registers a cleanup handler that triggers on script exit.

# ai_scientist/treesearch/perform_experiments_bfts_with_agentmanager.py

def cleanup():
    if global_step == 0:
        shutil.rmtree(cfg.workspace_dir)   # Remove unused experiment sandbox

atexit.register(cleanup)

If the experiment aborts before any progress is recorded (global_step == 0), the entire workspace directory is removed automatically, leaving no orphaned folders on the file system.

Summary

The AI-Scientist-v2 workspace isolation system provides comprehensive sandboxing through the following mechanisms:

• Unique top-level directories prevent cross-experiment contamination via incremental indexing combined with random slugs in prep_cfg
• Input and working subdirectories confine all file operations to the experiment's dedicated sandbox, separating data from execution artifacts
• Process-specific workspaces eliminate race conditions between parallel workers by creating process_{pid} subfolders with dedicated interpreters
• Automatic cleanup removes empty workspaces when experiments fail before generating output, maintaining file system hygiene

Frequently Asked Questions

How does AI-Scientist-v2 prevent parallel experiments from interfering with each other?

According to the ai_scientist/treesearch/parallel_agent.py source code, the system uses multiprocessing.current_process().name to create isolated subdirectories named process_{process_id} within each experiment workspace. Each parallel worker receives its own working directory and a dedicated Interpreter instance configured to that specific path, ensuring file operations from different processes never target the same location.

What happens to the workspace directory if an experiment fails immediately?

The perform_experiments_bfts_with_agentmanager.py file registers an atexit cleanup handler that checks if global_step == 0. If the experiment terminates before making any progress, the handler executes shutil.rmtree(cfg.workspace_dir) to delete the entire experiment sandbox, preventing empty directories from accumulating on the file system.

Where does the system store input datasets relative to the working directory?

As implemented in ai_scientist/treesearch/utils/config.py, the prep_agent_workspace function creates two subdirectories: input and working. The input folder contains a copy or symlink of the dataset specified in cfg.data_dir, while the working folder houses all generated code and temporary execution files. This separation ensures raw data remains immutable during the experiment.

Can multiple experiments run concurrently on the same machine without name collisions?

Yes. The prep_cfg function in config.py calculates the next available index by scanning both the log directory and workspace directory using _get_next_logindex, then formats the experiment name as f"{ind}-{cfg.exp_name}". This incremental indexing, combined with the optional random slug from coolname, guarantees unique directory names even when multiple instances launch simultaneously with identical configuration files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}