# How AI Scientist v2 Handles Experiment Parallelization: Architecture and Implementation

> Discover how AI Scientist v2 handles experiment parallelization by spawning Python processes with ParallelAgent and ProcessPoolExecutor for efficient multi-seed evaluations and exclusive GPU allocation.

- Repository: [Sakana AI/AI-Scientist-v2](https://github.com/SakanaAI/AI-Scientist-v2)
- Tags: architecture
- Published: 2026-03-28

---

**AI Scientist v2 executes experiments in parallel by spawning independent Python processes through the `ParallelAgent` class, utilizing `ProcessPoolExecutor` for worker management and `GPUManager` for exclusive GPU allocation, enabling concurrent multi-seed evaluations without blocking the main control loop.**

AI Scientist v2 from SakanaAI accelerates scientific discovery by running multiple experimental branches simultaneously through a sophisticated parallel execution engine. The system leverages process-based concurrency to evaluate diverse research ideas, tune hyperparameters, and validate results across random seeds. This article examines the source code in `SakanaAI/AI-Scientist-v2` to reveal how the framework orchestrates distributed experiments across CPU and GPU resources.

## Core Components of Parallel Execution

The parallelization engine centers on three integrated components that manage task distribution and hardware allocation.

### ParallelAgent Orchestrator

The **`ParallelAgent`** class in [`ai_scientist/treesearch/parallel_agent.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/ai_scientist/treesearch/parallel_agent.py) serves as the primary orchestrator for parallel search stages. It creates a pool of worker processes and distributes node-processing tasks including drafting, debugging, hyper-parameter tuning, and seed evaluation. The agent manages the full lifecycle of parallel execution from initialization through graceful shutdown.

### ProcessPoolExecutor Integration

The system utilizes Python's standard **`ProcessPoolExecutor`** from the `concurrent.futures` module to launch OS-level processes. In `ParallelAgent.__init__` (lines 1182-1184), the executor initializes with `max_workers` set to the configured number of parallel workers. This approach avoids Python's Global Interpreter Lock (GIL) by using separate process spaces rather than threads.

### GPUManager for Hardware Scheduling

The optional **`GPUManager`** class (lines 91-115 in [`parallel_agent.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/parallel_agent.py)) provides GPU-aware scheduling that assigns exclusive GPU identifiers to each worker process. When workers start, they receive a specific GPU ID via `CUDA_VISIBLE_DEVICES`, ensuring no resource contention between parallel experiments. If no GPUs are available, the system gracefully falls back to CPU execution.

## Configuration and Initialization

Enabling parallel mode requires specific configuration settings that activate the distributed execution pipeline.

### Enabling Parallel Mode in Configuration

The top-level configuration file controls parallel behavior through the `agent` section. Setting `agent.type` to `"parallel"` activates the parallel engine, while `agent.num_workers` defines the maximum number of concurrent processes.

```yaml

# config.yaml

agent:
  type: parallel          # Enables ParallelAgent

  num_workers: 4          # Maximum concurrent processes

  multi_seed_eval:
    num_seeds: 5          # Each node evaluated on 5 seeds in parallel

```

The configuration validation occurs in [`ai_scientist/treesearch/utils/config.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/ai_scientist/treesearch/utils/config.py) (lines 73-75), which ensures the `agent.type` field contains a valid execution mode.

### AgentManager Integration

For each stage in the experiment pipeline, the **`AgentManager._create_agent_for_stage`** method (lines 274-329 in [`agent_manager.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/agent_manager.py)) instantiates a `ParallelAgent` with stage-specific configuration. This factory pattern ensures each search stage receives the appropriate "best node" references and task descriptions required for distributed processing.

## Parallel Execution Workflow

The system follows a precise sequence to distribute work across processes and aggregate results.

### Worker Pool Setup

During initialization in `ParallelAgent.__init__` (lines 1171-1185), the system performs hardware detection and resource allocation:

```python
self.num_gpus = get_gpu_count()
self.gpu_manager = GPUManager(self.num_gpus) if self.num_gpus > 0 else None
if self.num_gpus > 0:
    self.num_workers = min(self.num_workers, self.num_gpus)
self.executor = ProcessPoolExecutor(max_workers=self.num_workers)

```

The code automatically caps the number of workers to match available GPUs when hardware acceleration is present, ensuring optimal resource utilization.

### Job Distribution and GPU Assignment

When evaluating multiple random seeds, `ParallelAgent._run_multi_seed_evaluation` (lines 1269-1300) constructs per-seed node copies and submits them to the process pool:

```python
futures.append(
    self.executor.submit(
        self._process_node_wrapper,
        node_data,
        self.task_desc,
        self.cfg,
        gpu_id,
        ...
    )
)

```

Before execution, each worker calls `GPUManager.acquire_gpu(process_id)` (lines 100-107) to obtain an exclusive GPU identifier. The worker then sets `CUDA_VISIBLE_DEVICES` to isolate that specific GPU:

```python
gpu_id = self.gpu_manager.acquire_gpu(process_id)
os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_id)

```

### Result Collection and Error Handling

The main thread awaits completion through `future.result(timeout=self.timeout)`, reconstructing `Node` objects from pickled return values (lines 1717-1730). Errors from individual workers are logged and isolated, preventing a single failed experiment from aborting the entire stage. Results append to a shared `Journal` that tracks the evolutionary search progress.

### Graceful Shutdown

Upon completion or exception, `ParallelAgent.__exit__` executes shutdown logic (lines 2339-2363) that calls `self.executor.shutdown(wait=False, cancel_futures=True)` and releases GPU assignments. This ensures system resources return to the pool even when experiments terminate unexpectedly.

## Practical Implementation Examples

### Configuring Parallel Experiments

Create a YAML configuration to activate parallel execution with GPU awareness:

```yaml

# config.yaml

data_dir: example_tasks/your_task
goal: "Optimize neural architecture for image classification"
agent:
  type: parallel               # Activate parallel execution

  num_workers: 8               # Use up to 8 processes

  code:
    model: gpt-4o-mini
    temp: 0.2
  multi_seed_eval:
    num_seeds: 5               # Parallel seed evaluations per node

```

### Running via AgentManager

Execute parallel experiments through the high-level API:

```python
from ai_scientist.treesearch.agent_manager import AgentManager
from ai_scientist.treesearch.utils.config import load_cfg

cfg = load_cfg()                     # Loads the YAML configuration

manager = AgentManager(cfg)          # Creates ParallelAgent for each stage

manager.run()                        # Orchestrates distributed search

```

### Manual Process Submission

For custom workflows, submit tasks directly to the process pool:

```python
from concurrent.futures import ProcessPoolExecutor
from ai_scientist.treesearch.parallel_agent import GPUManager, _process_node_wrapper
from ai_scientist.treesearch.utils.gpu import get_gpu_count

cfg = load_cfg()
gpu_manager = GPUManager(get_gpu_count())
executor = ProcessPoolExecutor(max_workers=4)

def submit_seed(node_dict, seed):
    proc_id = f"seed_{seed}"
    gpu = gpu_manager.acquire_gpu(proc_id) if gpu_manager else None
    return executor.submit(
        _process_node_wrapper,
        node_dict,
        "Task description",
        cfg,
        gpu,
        "",                     # memory_summary placeholder

        None, None, None, None, None, None,
        seed_eval=True,
    )

```

## Summary

- **AI Scientist v2** achieves experiment parallelization through the **`ParallelAgent`** class, which orchestrates multiple OS processes via `ProcessPoolExecutor`.
- The system automatically detects GPU resources and assigns exclusive devices to workers through **`GPUManager`**, falling back to CPU when GPUs are unavailable.
- Configuration occurs via `agent.type: parallel` in [`config.yaml`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/config.yaml), with `num_workers` controlling concurrency levels.
- The **`AgentManager`** factory creates stage-specific parallel agents, while `ParallelAgent._run_multi_seed_evaluation` distributes multi-seed experiments across the worker pool.
- Results aggregate through future callbacks that reconstruct node objects, with isolated error handling preventing cascade failures.
- Graceful shutdown in `ParallelAgent.__exit__` ensures resource cleanup via `executor.shutdown()` and GPU release.

## Frequently Asked Questions

### How does AI Scientist v2 distribute GPU resources across parallel workers?

The system uses the **`GPUManager`** class in [`parallel_agent.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/parallel_agent.py) to maintain a registry of available GPUs. When a worker process starts, it calls `acquire_gpu(process_id)` (lines 100-107) to receive an exclusive GPU identifier, which the worker sets as `CUDA_VISIBLE_DEVICES`. This ensures each process sees only its assigned GPU, preventing memory conflicts and maximizing throughput.

### What is the difference between parallel and sequential execution modes?

Sequential execution processes one experiment branch at a time through the `SequentialAgent`, while parallel mode spawns independent Python processes via `ParallelAgent`. According to [`utils/config.py`](https://github.com/SakanaAI/AI-Scientist-v2/blob/main/utils/config.py) (lines 73-75), setting `agent.type` to `"parallel"` activates the process pool, enabling concurrent evaluation of multiple seeds, drafts, or debugging iterations without blocking the main search loop.

### How does the system handle failures in individual parallel workers?

The main thread in `ParallelAgent` wraps each `future.result()` call (lines 1717-1730) in exception handling that logs errors without terminating the stage. Failed workers return error states that the system records in the `Journal`, allowing the evolutionary search to continue with remaining successful experiments while preserving error metadata for later analysis.

### Can AI Scientist v2 run parallel experiments on CPU-only machines?

Yes. During initialization in `ParallelAgent.__init__` (lines 1171-1185), the system checks `get_gpu_count()` and sets `self.gpu_manager = None` when no GPUs are detected. The `ProcessPoolExecutor` continues with CPU-based workers, though the `num_workers` cap logic only applies when GPUs are present, allowing full CPU concurrency as specified in the configuration.