Setting up Figure Generation with FigureAgent and Nano Banana in AutoResearchClaw

Enable automated figure creation in AutoResearchClaw by installing the Gemini SDK, exporting your GEMINI_API_KEY, and configuring FigureAgentConfig with nano_banana_enabled: true to generate both data-driven charts and conceptual diagrams via the FigureOrchestrator.

AutoResearchClaw (ARC) automates research paper creation through a multi-agent pipeline that produces publication-ready figures. The FigureAgent orchestrates this process by routing data-driven visualizations to a code-generation backend while delegating conceptual diagrams to NanoBanana, Google's Gemini image generation API. This guide demonstrates how to configure and execute the complete figure generation workflow using the actual source implementation in the aiming-lab/AutoResearchClaw repository.

Prerequisites and Dependencies

The figure generation pipeline requires optional dependencies for Nano Banana image generation. While data-driven charts rely only on standard scientific Python libraries, conceptual diagrams need the Gemini client.

Install the required packages:

pip install "google-genai>=0.5" Pillow

If you skip this step, ARC will attempt to use a built-in REST fallback for image generation, though the SDK path is preferred for reliability and additional features.

Configuring FigureAgent and Nano Banana

The behavior of the entire figure subsystem is governed by FigureAgentConfig defined in [researchclaw/config.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/config.py). This dataclass controls generation limits, retry logic, and API credentials.

Key configuration fields include:

• nano_banana_enabled: Boolean toggle to enable/disable Gemini image generation
• gemini_api_key: Your API key (falls back to GEMINI_API_KEY or GOOGLE_API_KEY environment variables)
• gemini_model: Defaults to "gemini-2.5-flash-image"
• min_figures and max_figures: Constraints passed to the decision agent (default 3–8)
• max_iterations: Retry attempts for the code generation pipeline (default 3)
• output_format: Either "python" for Matplotlib or "latex" for TikZ

Set your API key via environment variable:

export GEMINI_API_KEY="your-api-key-here"

# or

export GOOGLE_API_KEY="your-api-key-here"

Example YAML configuration:

figure_agent:
  enabled: true
  min_figures: 3
  max_figures: 6
  max_iterations: 2
  nano_banana_enabled: true
  gemini_api_key: ""  # Leave empty to use env var

  gemini_model: "gemini-2.5-flash-image"
  dpi: 300

Understanding the Figure Generation Pipeline

The FigureOrchestrator in [researchclaw/agents/figure_agent/orchestrator.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/agents/figure_agent/orchestrator.py) coordinates four distinct phases:

1. Decision: FigureDecisionAgent analyzes your draft and experiments to determine which figures are needed
2. Routing: Each figure is tagged with a backend—either "code" for data-driven charts or "image" for conceptual diagrams
3. Execution: Parallel pipelines generate assets (Matplotlib/TikZ for code, NanoBanana for images)
4. Integration: IntegratorAgent consolidates outputs into a FigurePlan manifest

Data-Driven Charts (Code Backend)

When FigureDecisionAgent assigns backend: "code" (defined as FIGURE_CATEGORY_DATA in [researchclaw/agents/figure_agent/decision.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/agents/figure_agent/decision.py)), the orchestrator executes a four-step chain:

• PlannerAgent: Designs the visualization strategy
• CodeGenAgent: Generates Python (Matplotlib) or LaTeX (TikZ) code
• RendererAgent: Executes the code in a sandboxed environment (Docker optional)
• CriticAgent: Validates output and triggers retries if rendering fails

This loop repeats up to max_iterations times until valid output is produced or the limit is reached.

Conceptual Diagrams (Image Backend)

For figures requiring architectural diagrams or flowcharts (backend: "image"), the orchestrator delegates to NanoBananaAgent in [researchclaw/agents/figure_agent/nano_banana.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/agents/figure_agent/nano_banana.py).

The agent constructs prompts using _build_prompt(), combining your figure description with academic style guidelines and aspect ratio constraints. It then calls the Gemini API using either the SDK or REST fallback, returning PNG files with metadata including figure_id, output_path, and caption.

Running the Orchestrator

Invoke the pipeline programmatically by instantiating FigureOrchestrator with your LLM client and configuration:

from pathlib import Path
from researchclaw.agents.figure_agent.orchestrator import (
    FigureOrchestrator, 
    FigureAgentConfig
)
from researchclaw.llms.fake_llm import FakeLLM  # Replace with your LLM wrapper

# 1. Initialize your LLM

llm = FakeLLM()  # Must implement .chat(messages, **kwargs)

# 2. Configure the agent

cfg = FigureAgentConfig(
    min_figures=3,
    max_figures=5,
    max_iterations=2,
    nano_banana_enabled=True,  # Enable Nano Banana

    gemini_api_key=None,       # Reads from GEMINI_API_KEY env var

)

# 3. Create orchestrator with debug staging

orch = FigureOrchestrator(llm, cfg, stage_dir=Path("./stage-figures"))

# 4. Execute with research context

plan = orch.orchestrate({
    "topic": "Neural Architecture Search for Vision Transformers",
    "hypothesis": "Our NAS method yields higher accuracy with fewer FLOPs.",
    "paper_draft": "# Introduction\n...",

    "experiment_results": {"accuracy": 0.94, "flops": 1.2e9},
    "condition_summaries": {"baseline": "ResNet-50"},
    "metrics_summary": {"primary": "accuracy"},
    "metric_key": "accuracy",
    "output_dir": "./charts",
})

# 5. Inspect results

print(f"Generated {plan.figure_count} figures")
print(f"Manifest: {plan.manifest_path}")

The orchestrator returns a FigurePlan object containing:

• manifest: JSON list of figure metadata with file paths and captions
• markdown_refs: Pre-formatted markdown image links ready for insertion
• figure_descriptions: Textual summaries for the paper-writing LLM
• output_dir: Path to generated assets

Consuming FigurePlan in Downstream Stages

After generation, the paper writing stage consumes the FigurePlan to insert references and descriptions. In [researchclaw/pipeline/stage_impls/_paper_writing.py](https://github.com/aiming-lab/AutoResearchClaw/blob/main/researchclaw/pipeline/stage_impls/_paper_writing.py), the draft prompt is augmented with figure descriptions:

if figure_plan := context.get("figure_plan"):
    draft_prompt = f"{draft_prompt}\n\n{figure_plan.figure_descriptions}"

Example decision JSON from FigureDecisionAgent showing mixed backends:

[
  {
    "section": "Method",
    "figure_type": "architecture_diagram",
    "backend": "image",
    "description": "Overview of the proposed model architecture showing encoder-decoder blocks.",
    "priority": 1
  },
  {
    "section": "Results",
    "figure_type": "bar_comparison",
    "backend": "code",
    "description": "Bar chart comparing top-1 accuracy vs. baselines on ImageNet.",
    "priority": 1
  }
]

Summary

• Install dependencies with pip install "google-genai>=0.5" Pillow to enable Nano Banana image generation
• Configure FigureAgentConfig in researchclaw/config.py, ensuring nano_banana_enabled: true and valid gemini_api_key (or environment variable)
• Route automatically via FigureDecisionAgent, which assigns "code" backends to data charts and "image" backends to conceptual diagrams
• Execute using FigureOrchestrator.orchestrate() with your research context to generate both Matplotlib/TikZ figures and Gemini-generated PNGs
• Consume the resulting FigurePlan manifest in downstream paper writing stages via markdown_refs and figure_descriptions

Frequently Asked Questions

What is the difference between the code and image backends?

The code backend (FIGURE_CATEGORY_DATA) generates data-driven charts like bar plots and line graphs using Python (Matplotlib) or LaTeX (TikZ) code executed by the RendererAgent. The image backend (FIGURE_CATEGORY_IMAGE) creates conceptual diagrams like architecture flows or system overviews using NanoBananaAgent to call Google's Gemini image API. The FigureDecisionAgent automatically selects the appropriate backend based on figure type and content requirements.

Can I use Nano Banana without installing the google-genai SDK?

Yes. While installing google-genai>=0.5 provides the most reliable integration, NanoBananaAgent includes a REST fallback that calls https://generativelanguage.googleapis.com/v1beta/models/ directly using standard library urllib calls. This requires no additional packages but needs the GEMINI_API_KEY environment variable set.

How are generated figures inserted into the final research paper?

The IntegratorAgent produces a FigurePlan object containing markdown_refs (formatted image links) and figure_descriptions (textual summaries). The paper writing stage in _paper_writing.py retrieves this plan from the pipeline context and injects the descriptions into the LLM prompt, while the markdown references are inserted into the final LaTeX or Markdown document during the compilation phase.

What happens if figure generation fails?

For code backend figures, the CriticAgent validates each rendering attempt, and the orchestrator retries up to max_iterations times (default 3) if code execution fails or outputs are malformed. For image backend figures, Nano Banana will skip generation if the API key is missing or the call times out, logging a warning while allowing the pipeline to continue with other figures. Enable strict_mode: true in FigureAgentConfig to treat failures as fatal errors rather than warnings.

How do I customize the aspect ratio for Nano Banana images?

The NanoBananaAgent supports aspect ratio configuration through the generationConfig parameter in its REST API calls or SDK requests. Modify the aspectRatio field (e.g., "16:9", "1:1", "4:3") in the request payload built within _build_request() in nano_banana.py to match your publication requirements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}