How AI Scientist v2 Handles Experiment Parallelization: Architecture and Implementation

AI Scientist v2 executes experiments in parallel by spawning independent Python processes through the ParallelAgent class, utilizing ProcessPoolExecutor for worker management and GPUManager for exclusive GPU allocation, enabling concurrent multi-seed evaluations without blocking the main control loop.

AI Scientist v2 from SakanaAI accelerates scientific discovery by running multiple experimental branches simultaneously through a sophisticated parallel execution engine. The system leverages process-based concurrency to evaluate diverse research ideas, tune hyperparameters, and validate results across random seeds. This article examines the source code in SakanaAI/AI-Scientist-v2 to reveal how the framework orchestrates distributed experiments across CPU and GPU resources.

Core Components of Parallel Execution

The parallelization engine centers on three integrated components that manage task distribution and hardware allocation.

ParallelAgent Orchestrator

The ParallelAgent class in ai_scientist/treesearch/parallel_agent.py serves as the primary orchestrator for parallel search stages. It creates a pool of worker processes and distributes node-processing tasks including drafting, debugging, hyper-parameter tuning, and seed evaluation. The agent manages the full lifecycle of parallel execution from initialization through graceful shutdown.

ProcessPoolExecutor Integration

The system utilizes Python's standard ProcessPoolExecutor from the concurrent.futures module to launch OS-level processes. In ParallelAgent.__init__ (lines 1182-1184), the executor initializes with max_workers set to the configured number of parallel workers. This approach avoids Python's Global Interpreter Lock (GIL) by using separate process spaces rather than threads.

GPUManager for Hardware Scheduling

The optional GPUManager class (lines 91-115 in parallel_agent.py) provides GPU-aware scheduling that assigns exclusive GPU identifiers to each worker process. When workers start, they receive a specific GPU ID via CUDA_VISIBLE_DEVICES, ensuring no resource contention between parallel experiments. If no GPUs are available, the system gracefully falls back to CPU execution.

Configuration and Initialization

Enabling parallel mode requires specific configuration settings that activate the distributed execution pipeline.

Enabling Parallel Mode in Configuration

The top-level configuration file controls parallel behavior through the agent section. Setting agent.type to "parallel" activates the parallel engine, while agent.num_workers defines the maximum number of concurrent processes.

# config.yaml

agent:
  type: parallel          # Enables ParallelAgent

  num_workers: 4          # Maximum concurrent processes

  multi_seed_eval:
    num_seeds: 5          # Each node evaluated on 5 seeds in parallel

The configuration validation occurs in ai_scientist/treesearch/utils/config.py (lines 73-75), which ensures the agent.type field contains a valid execution mode.

AgentManager Integration

For each stage in the experiment pipeline, the AgentManager._create_agent_for_stage method (lines 274-329 in agent_manager.py) instantiates a ParallelAgent with stage-specific configuration. This factory pattern ensures each search stage receives the appropriate "best node" references and task descriptions required for distributed processing.

Parallel Execution Workflow

The system follows a precise sequence to distribute work across processes and aggregate results.

Worker Pool Setup

During initialization in ParallelAgent.__init__ (lines 1171-1185), the system performs hardware detection and resource allocation:

self.num_gpus = get_gpu_count()
self.gpu_manager = GPUManager(self.num_gpus) if self.num_gpus > 0 else None
if self.num_gpus > 0:
    self.num_workers = min(self.num_workers, self.num_gpus)
self.executor = ProcessPoolExecutor(max_workers=self.num_workers)

The code automatically caps the number of workers to match available GPUs when hardware acceleration is present, ensuring optimal resource utilization.

Job Distribution and GPU Assignment

When evaluating multiple random seeds, ParallelAgent._run_multi_seed_evaluation (lines 1269-1300) constructs per-seed node copies and submits them to the process pool:

futures.append(
    self.executor.submit(
        self._process_node_wrapper,
        node_data,
        self.task_desc,
        self.cfg,
        gpu_id,
        ...
    )
)

Before execution, each worker calls GPUManager.acquire_gpu(process_id) (lines 100-107) to obtain an exclusive GPU identifier. The worker then sets CUDA_VISIBLE_DEVICES to isolate that specific GPU:

gpu_id = self.gpu_manager.acquire_gpu(process_id)
os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_id)

Result Collection and Error Handling

The main thread awaits completion through future.result(timeout=self.timeout), reconstructing Node objects from pickled return values (lines 1717-1730). Errors from individual workers are logged and isolated, preventing a single failed experiment from aborting the entire stage. Results append to a shared Journal that tracks the evolutionary search progress.

Graceful Shutdown

Upon completion or exception, ParallelAgent.__exit__ executes shutdown logic (lines 2339-2363) that calls self.executor.shutdown(wait=False, cancel_futures=True) and releases GPU assignments. This ensures system resources return to the pool even when experiments terminate unexpectedly.

Practical Implementation Examples

Configuring Parallel Experiments

Create a YAML configuration to activate parallel execution with GPU awareness:

# config.yaml

data_dir: example_tasks/your_task
goal: "Optimize neural architecture for image classification"
agent:
  type: parallel               # Activate parallel execution

  num_workers: 8               # Use up to 8 processes

  code:
    model: gpt-4o-mini
    temp: 0.2
  multi_seed_eval:
    num_seeds: 5               # Parallel seed evaluations per node

Running via AgentManager

Execute parallel experiments through the high-level API:

from ai_scientist.treesearch.agent_manager import AgentManager
from ai_scientist.treesearch.utils.config import load_cfg

cfg = load_cfg()                     # Loads the YAML configuration

manager = AgentManager(cfg)          # Creates ParallelAgent for each stage

manager.run()                        # Orchestrates distributed search

Manual Process Submission

For custom workflows, submit tasks directly to the process pool:

from concurrent.futures import ProcessPoolExecutor
from ai_scientist.treesearch.parallel_agent import GPUManager, _process_node_wrapper
from ai_scientist.treesearch.utils.gpu import get_gpu_count

cfg = load_cfg()
gpu_manager = GPUManager(get_gpu_count())
executor = ProcessPoolExecutor(max_workers=4)

def submit_seed(node_dict, seed):
    proc_id = f"seed_{seed}"
    gpu = gpu_manager.acquire_gpu(proc_id) if gpu_manager else None
    return executor.submit(
        _process_node_wrapper,
        node_dict,
        "Task description",
        cfg,
        gpu,
        "",                     # memory_summary placeholder

        None, None, None, None, None, None,
        seed_eval=True,
    )

Summary

• AI Scientist v2 achieves experiment parallelization through the ParallelAgent class, which orchestrates multiple OS processes via ProcessPoolExecutor.
• The system automatically detects GPU resources and assigns exclusive devices to workers through GPUManager, falling back to CPU when GPUs are unavailable.
• Configuration occurs via agent.type: parallel in config.yaml, with num_workers controlling concurrency levels.
• The AgentManager factory creates stage-specific parallel agents, while ParallelAgent._run_multi_seed_evaluation distributes multi-seed experiments across the worker pool.
• Results aggregate through future callbacks that reconstruct node objects, with isolated error handling preventing cascade failures.
• Graceful shutdown in ParallelAgent.__exit__ ensures resource cleanup via executor.shutdown() and GPU release.

Frequently Asked Questions

How does AI Scientist v2 distribute GPU resources across parallel workers?

The system uses the GPUManager class in parallel_agent.py to maintain a registry of available GPUs. When a worker process starts, it calls acquire_gpu(process_id) (lines 100-107) to receive an exclusive GPU identifier, which the worker sets as CUDA_VISIBLE_DEVICES. This ensures each process sees only its assigned GPU, preventing memory conflicts and maximizing throughput.

What is the difference between parallel and sequential execution modes?

Sequential execution processes one experiment branch at a time through the SequentialAgent, while parallel mode spawns independent Python processes via ParallelAgent. According to utils/config.py (lines 73-75), setting agent.type to "parallel" activates the process pool, enabling concurrent evaluation of multiple seeds, drafts, or debugging iterations without blocking the main search loop.

How does the system handle failures in individual parallel workers?

The main thread in ParallelAgent wraps each future.result() call (lines 1717-1730) in exception handling that logs errors without terminating the stage. Failed workers return error states that the system records in the Journal, allowing the evolutionary search to continue with remaining successful experiments while preserving error metadata for later analysis.

Can AI Scientist v2 run parallel experiments on CPU-only machines?

Yes. During initialization in ParallelAgent.__init__ (lines 1171-1185), the system checks get_gpu_count() and sets self.gpu_manager = None when no GPUs are detected. The ProcessPoolExecutor continues with CPU-based workers, though the num_workers cap logic only applies when GPUs are present, allowing full CPU concurrency as specified in the configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}