How to Configure Tree Search Parameters in AI Scientist v2: 3 Key Settings Explained

To configure tree search parameters in AI Scientist v2, adjust the max_debug_depth, debug_prob, and num_drafts values in the agent.search section of your YAML configuration or override them via command-line arguments when launching experiments.

The SakanaAI/AI-Scientist-v2 repository implements a best-first tree search algorithm that autonomously explores research ideas through iterative drafting and debugging. Configuring these tree search parameters allows you to control the trade-off between exploring novel concepts and refining existing implementations that contain bugs.

Understanding the Core Tree Search Parameters

AI Scientist v2 exposes three hyper-parameters that govern the search behavior. These are defined in the SearchConfig dataclass located in ai_scientist/treesearch/utils/config.py (lines 44-49) and stored under the agent.search key in your configuration file.

• max_debug_depth (integer, default: 3): Controls the maximum tree depth at which a node remains eligible for the debug phase. Nodes deeper than this value in ai_scientist/treesearch/parallel_agent.py are never selected for bug-fixing attempts, preventing the agent from spending resources on overly complex nested issues.

• debug_prob (float between 0 and 1, default: 0.5): Determines the probability that the search enters the debug phase during each iteration. Lower values prioritize generating new research drafts, while higher values allocate more cycles to fixing existing buggy implementations.

• num_drafts (integer, default: 3): Specifies how many draft nodes the agent must generate before transitioning to the normal parallel search phase. Increasing this value expands the initial candidate pool at the cost of additional computation time.

The default values for these parameters reside in bfts_config.yaml within the agent.search section (lines 73-77).

Modifying Configuration via YAML

The primary method for configuring tree search parameters involves editing your experiment configuration file. The repository provides bfts_config.yaml as the default template.

Create or edit your YAML file to customize the search behavior:

agent:
  type: parallel
  num_workers: 4
  search:
    max_debug_depth: 5      # Allow debugging up to depth 5

    debug_prob: 0.3         # 30% chance to enter debug mode per iteration

    num_drafts: 6           # Generate six draft nodes before main search

When the experiment launches via launch_scientist_bfts.py, the configuration loader parses these values into the Config dataclass, making them accessible throughout the codebase as cfg.agent.search.

Overriding Parameters via Command Line

For rapid experimentation without editing files, AI Scientist v2 supports command-line overrides using OmegaConf syntax. The load_cfg() function in ai_scientist/treesearch/utils/config.py (lines 28-31) automatically merges CLI arguments with YAML defaults.

Pass parameters directly when invoking the launch script:

python launch_scientist_bfts.py \
  --agent.search.max_debug_depth=4 \
  --agent.search.debug_prob=0.2 \
  --agent.search.num_drafts=5

These arguments take precedence over YAML values and immediately affect the runtime behavior of the parallel agent defined in ai_scientist/treesearch/parallel_agent.py.

How Parameters Impact the Search Process

Understanding the runtime effects of each parameter helps optimize your research workflow.

Increasing max_debug_depth allows the agent to attempt fixes on deeply nested nodes, potentially resolving sophisticated bugs in complex implementations. However, deeper nodes typically carry more state overhead, which may increase computational costs and experiment duration.

Adjusting debug_prob directly controls the exploration-exploitation trade-off. Setting this to 0.0 effectively disables the debug phase, forcing the agent to continuously draft new ideas without attempting repairs. Conversely, setting it to 1.0 causes the agent to focus exclusively on debugging existing nodes.

Modifying num_drafts changes the breadth of the initial exploration phase. Higher values improve diversity in the candidate pool before parallel evaluation begins, though they delay the start of the main search loop.

Code Examples

Accessing Search Configuration in Python

When extending the codebase or writing custom evaluation scripts, access the tree search parameters through the configuration object:

from pathlib import Path
from ai_scientist.treesearch.utils.config import load_cfg

cfg = load_cfg(Path("custom_config.yaml"))
search_cfg = cfg.agent.search

print(f"Debug depth limit: {search_cfg.max_debug_depth}")
print(f"Debug probability: {search_cfg.debug_prob}")
print(f"Required drafts: {search_cfg.num_drafts}")

# Runtime decision making based on debug probability

import random
if random.random() < search_cfg.debug_prob:
    # Filter nodes eligible for debugging

    debuggable = [
        n for n in journal.buggy_nodes
        if n.is_leaf and n.debug_depth <= search_cfg.max_debug_depth
    ]

This pattern appears in ai_scientist/treesearch/parallel_agent.py (lines 44-66), where the agent uses these values to select nodes for processing.

Complete Custom Configuration File

Define a standalone configuration for a specific experiment:

data_dir: "data"
log_dir: "logs"
workspace_dir: "workspaces"

agent:
  type: parallel
  num_workers: 8
  search:
    max_debug_depth: 4
    debug_prob: 0.25
    num_drafts: 10

Save this as custom_bfts_config.yaml and reference it when launching your experiment.

Summary

• Tree search parameters in AI Scientist v2 are controlled through max_debug_depth, debug_prob, and num_drafts under the agent.search configuration key.
• Default values reside in bfts_config.yaml, with schema validation handled by the SearchConfig dataclass in ai_scientist/treesearch/utils/config.py.
• Configuration methods include editing YAML files or passing --agent.search.<parameter>=<value> arguments via the command line.
• Runtime impact varies by parameter: depth controls debugging eligibility, probability balances drafting versus fixing, and draft count sets initial exploration breadth.
• Implementation reference shows these values being consumed in ai_scientist/treesearch/parallel_agent.py to filter buggy_nodes and determine search phase transitions.

Frequently Asked Questions

What are the default values for tree search parameters in AI Scientist v2?

The default configuration in bfts_config.yaml sets max_debug_depth to 3, debug_prob to 0.5, and num_drafts to 3. These values provide a balanced approach between exploring new ideas and fixing implementation bugs.

Can I disable the debug phase entirely in AI Scientist v2?

Yes. Set debug_prob to 0.0 in your configuration file or via command line (--agent.search.debug_prob=0.0). This prevents the search from entering the debug phase, causing the agent to focus exclusively on generating new research drafts without attempting to fix buggy implementations.

How does max_debug_depth affect experiment runtime?

Increasing max_debug_depth allows the agent to debug deeper nodes in the search tree, which often contain more complex state and require additional computational resources to process. While this may lead to more sophisticated solutions, it typically increases the time required to complete experimental iterations.

Where is the SearchConfig dataclass defined in the source code?

The SearchConfig dataclass is defined in ai_scientist/treesearch/utils/config.py at lines 44-49. This class maps the YAML configuration fields to typed Python attributes and is accessed throughout the tree search implementation, particularly in ai_scientist/treesearch/parallel_agent.py where the actual search logic executes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}