# How to Use SpiderFoot's Built-In Data Visualizations: A Complete Guide for Security Analysts

> Unlock SpiderFoot's built-in data visualizations. Learn how security analysts can leverage interactive D3.js charts to understand scan relationships and enhance investigations through our complete guide.

- Repository: [Steve Micallef/spiderfoot](https://github.com/smicallef/spiderfoot)
- Tags: how-to-guide
- Published: 2026-08-15

---

**SpiderFoot converts raw scan relationships into interactive D3.js charts using the `dataParentChildToTree` helper in [`spiderfoot/helpers.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/helpers.py), exposing them through the "Visualisations" tab in the web UI.**

SpiderFoot's reconnaissance platform doesn't just collect OSINT data—it transforms discovered relationships into visual graphs that reveal hidden infrastructure patterns. This guide explains how SpiderFoot's built-in data visualizations work, how to access them through the web interface, and how to reuse the visualization logic in your own tools.

## How SpiderFoot Data Visualizations Work

SpiderFoot's visualization pipeline follows a clean separation between data extraction and rendering. Understanding this architecture helps you troubleshoot issues or extend the system.

### The Core Conversion Helper

All visualizations rely on **`SpiderFootHelpers.dataParentChildToTree`** in [`spiderfoot/helpers.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/helpers.py) (lines 363-391). This static method takes a flat parent-child dictionary and returns a nested JSON structure that D3.js expects:

- Each node has a **`name`** string
- Each node has an optional **`children`** array (or `None` for leaves)
- The method validates input and guarantees a single root node

The helper enforces type safety: invalid input raises `TypeError` or `ValueError`, preventing malformed data from reaching the front end.

### The End-to-End Flow

1. **Scan execution** stores entity relationships in SpiderFoot's SQLite database
2. **Web UI request** hits the `/visualise` endpoint (handled in [`sfwebui.py`](https://github.com/smicallef/spiderfoot/blob/main/sfwebui.py))
3. **Backend processing** queries the database, assembles a parent → [children] mapping
4. **Conversion** passes the mapping to `dataParentChildToTree`
5. **JSON delivery** sends the nested structure to the browser
6. **D3 rendering** draws the interactive visualization

This pipeline means any SpiderFoot scan—launched via web UI, CLI ([`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py)), or API—automatically supports visualization once complete.

## Built-In Visualization Types

SpiderFoot's web UI exposes three primary chart types through the **Visualisations** tab. Each serves different investigative purposes.

### Domain Graph

**What it shows:** DNS hierarchy including domains, subdomains, and name servers

**Source data:** Parent-child relationships from DNS record modules

**How it's built:** 
- `dataParentChildToTree` creates a D3-compatible tree structure
- Front-end renders with `d3-tree` layout
- Natural top-down reading matches DNS delegation chains

Use this when tracking domain infrastructure or identifying unexpected subdomains.

### Network Graph

**What it shows:** IP address and host relationships—who hosts what

**Source data:** IP ↔ hostname mappings stored during scanning

**How it's built:**
- Same helper generates the JSON payload
- UI applies a **force-directed D3 layout** instead of a tree
- Physics simulation clusters related hosts naturally

This view excels at identifying shared hosting infrastructure and CDN usage patterns.

### Entity Relationship Diagram

**What it shows:** Multi-hop connections between entities (emails, persons, companies, domains)

**Source data:** Entity-entity edges from correlation modules

**How it's built:**
- Helper constructs a tree where each entity becomes a node
- Relationships become parent-child or sibling connections
- D3 renders as an explorable network graph

Use this for social network analysis and supply chain mapping.

## Accessing Visualizations Through the Web UI

The simplest way to use SpiderFoot's data visualizations is through the built-in web interface.

### Step-by-Step: View Your First Graph

1. **Start SpiderFoot** (Docker: `docker run -p 5001:5001 spiderfoot`, or local install)
2. **Navigate to** `http://localhost:5001`
3. **Create or select a scan** and run it to completion
4. **Click the "Visualisations" tab** in the scan results
5. **Select your chart type** (Domain Graph, Network Graph, or Entity Relationship)
6. **Interact** with the rendered D3 diagram—zoom, pan, and click nodes to explore

Behind the scenes, the UI issues a GET request to:

```

/visualise?type=domaingraph&scan_id=<SCAN_ID>

```

The `type` parameter selects the visualization variant; `scan_id` targets your specific scan data. No additional configuration is required—the backend automatically builds the appropriate relationship map from the database.

## Generating Visualization Data in Python

You can reuse SpiderFoot's conversion logic outside the bundled UI. This enables custom dashboards, automated reporting, or integration with other analysis tools.

### Stand-Alone Tree Generation

```python
from spiderfoot.helpers import SpiderFootHelpers

# Example: DNS parent-child mapping from your own data source

dns_map = {
    "example.com": ["ns1.example.com", "ns2.example.com"],
    "ns1.example.com": ["192.0.2.1"],
    "ns2.example.com": ["192.0.2.2"],
    "192.0.2.1": None,
    "192.0.2.2": None,
}

# Convert to D3-compatible nested structure

tree = SpiderFootHelpers.dataParentChildToTree(dns_map)
print(tree)

```

**Output structure:**

```json
{
  "name": "example.com",
  "children": [
    {
      "name": "ns1.example.com",
      "children": [
        {"name": "192.0.2.1", "children": null}
      ]
    },
    {
      "name": "ns2.example.com",
      "children": [
        {"name": "192.0.2.2", "children": null}
      ]
    }
  ]
}

```

The helper handles edge cases automatically:
- Validates that input is a dictionary
- Guarantees exactly one root node
- Recursively nests children to arbitrary depth
- Raises `TypeError` or `ValueError` for malformed input

## Embedding SpiderFoot Visualizations in Custom Pages

Extract the JSON from SpiderFoot's API and render it with your own D3 configuration.

### Minimal Custom Renderer

```html
<!DOCTYPE html>
<html>
<head>
  <script src="https://d3js.org/d3.v5.min.js"></script>
</head>
<body>
  <div id="chart"></div>

  <script>
    fetch(`/visualise?type=domaingraph&scan_id=<SCAN_ID>`)
      .then(response => response.json())
      .then(data => {
        const root = d3.hierarchy(data);
        const treeLayout = d3.tree().size([800, 600]);
        treeLayout(root);

        const svg = d3.select('#chart')
          .append('svg')
          .attr('width', 800)
          .attr('height', 600);

        // Draw links
        svg.selectAll('.link')
          .data(root.links())
          .enter()
          .append('line')
          .classed('link', true)
          .attr('x1', d => d.source.y)
          .attr('y1', d => d.source.x)
          .attr('x2', d => d.target.y)
          .attr('y2', d => d.target.x)
          .attr('stroke', '#ccc');

        // Draw nodes
        const node = svg.selectAll('.node')
          .data(root.descendants())
          .enter()
          .append('g')
          .classed('node', true)
          .attr('transform', d => `translate(${d.y},${d.x})`);

        node.append('circle').attr('r', 4);
        node.append('text')
          .attr('dx', 6)
          .attr('dy', 3)
          .text(d => d.data.name);
      });
  </script>
</body>
</html>

```

Replace `<SCAN_ID>` with your actual scan identifier. The fetched JSON format matches exactly what SpiderFoot's built-in UI consumes—you're simply applying custom D3 styling and layout choices.

## Key Implementation Files

| File | Purpose | Critical Lines |
|------|---------|----------------|
| [`spiderfoot/helpers.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/helpers.py) | Core conversion logic | `SpiderFootHelpers.dataParentChildToTree` at lines 363-391 |
| [`sfwebui.py`](https://github.com/smicallef/spiderfoot/blob/main/sfwebui.py) | HTTP endpoint and database queries | `/visualise` route handler |
| [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) | CLI entry point that populates visualization data | Scan execution flow |
| [`templates/visualisations.html`](https://github.com/smicallef/spiderfoot/blob/main/templates/visualisations.html) | D3.js front-end rendering | Flask-served template (implicit) |

Understanding these files lets you trace any visualization issue from UI symptom back to data source.

## Summary

- **SpiderFoot's data visualizations** rely on a single helper method—`dataParentChildToTree` in [`spiderfoot/helpers.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/helpers.py)—to convert flat relationship data into nested D3-compatible JSON
- **Three built-in chart types** cover DNS hierarchies (Domain Graph), infrastructure mapping (Network Graph), and multi-entity correlations (Entity Relationship)
- **Access via web UI** requires only completing a scan and clicking the Visualisations tab; no configuration needed
- **Reuse outside SpiderFoot** by importing `SpiderFootHelpers` or fetching `/visualise` endpoints for custom dashboards
- **All validation and tree-building logic** is centralized, making the system robust and extensible

## Frequently Asked Questions

### Does SpiderFoot require external tools to generate visualizations?

No. SpiderFoot bundles D3.js and handles the entire pipeline internally. The `dataParentChildToTree` helper produces JSON that any D3 installation can render, but SpiderFoot's web UI includes everything needed for interactive charts without additional dependencies.

### Can I visualize data from a scan that ran via CLI instead of the web UI?

Yes. SpiderFoot stores all scan results in the same SQLite database regardless of launch method. The `/visualise` endpoint reads from this database, so CLI-initiated scans appear in the web UI's Visualisations tab as long as you use the same database file.

### What happens if my scan data contains cycles or multiple root nodes?

The `dataParentChildToTree` helper enforces a strict tree structure. It validates input and raises `ValueError` if the mapping cannot resolve to a single root. In practice, SpiderFoot's data collection typically produces valid hierarchies; cycles in entity relationships are handled by the Entity Relationship diagram's force-directed layout rather than strict tree rendering.

### Is there a way to export visualization data for offline analysis?

Yes. Query the `/visualise` endpoint directly—the returned JSON is self-contained and portable. You can also use the Python helper to generate trees from exported scan data, then feed the result to any D3-compatible tool or save as a standard JSON file for documentation.