How to Implement Subnet and ASN Scanning in SpiderFoot: A Complete Technical Guide

SpiderFoot implements subnet and ASN scanning through three core components: the SpiderFootTarget class for target handling, a database schema with event types like NETBLOCK_OWNER and BGP_AS_OWNER, and modular scanning engines like sfp_ripe.py that expand ASNs into owned prefixes.

This guide walks through the architecture and implementation patterns for subnet and ASN scanning in the smicallef/spiderfoot open-source reconnaissance framework. Whether you're extending existing modules or building new ASN data sources, understanding these patterns is essential for effective network footprinting.

The Three-Pillar Architecture for Subnet and ASN Scanning

SpiderFoot's subnet and ASN support is built on three interconnected systems that work together during scan execution.

Target Handling with SpiderFootTarget

The SpiderFootTarget class in spiderfoot/target.py is the foundation. It stores the original target, maintains aliases, and provides subnet containment logic via the matches() method.

Key responsibilities include:

• Validating target types (NETBLOCK_OWNER, NETBLOCKV6_OWNER, BGP_AS_OWNER)
• Registering discovered subnets as aliases via setAlias()
• Testing whether discovered IPs fall within target netblocks

The subnet containment logic (around lines 5021-5027) uses netaddr for precise matching:

netaddr.IPAddress(value) in netaddr.IPNetwork(self.targetValue)

This allows the correlation engine to link any IP back to its parent netblock automatically.

Database Schema for Network Events

The event type definitions in spiderfoot/db.py drive the UI and correlation pipeline. Relevant types include:

• NETBLOCK_OWNER — IPv4 netblocks owned by the target
• NETBLOCKV6_OWNER — IPv6 netblocks
• BGP_AS_OWNER — Autonomous System registrations

These types determine which modules activate during scanning and how relationships display in results.

Modular Scanning Engine

Modules like sfp_ripe.py perform the actual ASN-to-prefix resolution. The pattern follows a clear sequence: receive an ASN event, query external APIs, expand to netblocks, and emit ownership events for downstream processing.

Command-Line Subnet and ASN Scanning

Scanning a Subnet Directly

Launch a netblock scan using the CLI with explicit target type specification:

# Scan the 10.0.0.0/16 netblock

sf.py -s 10.0.0.0/16 -t NETBLOCK_OWNER

The sf.py front-end parses -t NETBLOCK_OWNER, instantiates SpiderFootTarget with targetType="NETBLOCK_OWNER" and targetValue="10.0.0.0/16". Modules with netblocklookup=True (VirusTotal, Shodan, and others) automatically enumerate IPs in that range, respecting each plugin's maxnetblock size limits.

Scanning by ASN

For ASN-based reconnaissance:

# Scan Google's ASN and all owned prefixes

sf.py -s AS15169 -t BGP_AS_OWNER

The sfp_ripe module (or alternatives like sfp_bgpview) processes the ASN event, queries the RIPE API, and for each returned prefix executes:

self.sf.target.setAlias(prefix, "NETBLOCK_OWNER")
self.sf.emitEvent(prefix, "NETBLOCK_OWNER", self.__name__, asn_event)

This dual action—alias registration and event emission—ensures the prefix becomes both a queryable alias of the original target and a seed for further scanning.

Implementing Subnet Lookups in Custom Modules

Enabling Subnet Enumeration

Plugins that need to iterate IPs within subnets declare this capability through option configuration. From sfp_virustotal.py:

self.opts = {
    'subnetlookup': True,          # Enable enumeration of subnet IPs

    'maxsubnet': 24,               # Limit to /24 or smaller only

}

Runtime Subnet Validation

During execution, modules validate subnet size before expansion:

from netaddr import IPNetwork

if not self.opts['subnetlookup']:
    return

network = IPNetwork(eventData)
if network.prefixlen < self.opts['maxsubnet']:
    self.debug(f"Network size {network.prefixlen} exceeds max {self.opts['maxsubnet']}")
    return

# Proceed with IP enumeration

This pattern prevents unbounded scans on large allocations while allowing focused probing of manageable subnets.

Matching Discovered Hosts to Targets

Use SpiderFootTarget.matches() to verify IP containment:

if self.sf.target.matches(ip_address):
    self.info(f"{ip_address} belongs to original target subnet")

This method implements the netaddr containment check, ensuring proper relationship mapping in results.

The Alias Mechanism: Linking ASNs to Subnets

The setAlias() method is critical for multi-level target expansion. When an ASN module discovers a subnet:

1. The subnet is registered as an alias via target.setAlias(subnet, "NETBLOCK_OWNER")
2. The subnet receives its own NETBLOCK_OWNER event
3. Subsequent correlation treats the subnet as part of the original scan scope
4. The correlation engine in spiderfoot/correlation.py (lines 628-633) applies match_method='subnet' rules to link IP events to their parent netblocks

This alias chain enables complex queries like "show all IPs discovered from any subnet of ASN X."

Building a New ASN Module: Step-by-Step

Follow this pattern to add support for new ASN data sources:

1. Inherit from BaseModule

from spiderfoot import SpiderFootPlugin, SpiderFootEvent

class MyASNModule(SpiderFootPlugin):
    meta = {
        'name': 'My ASN Source',
        'events': ['BGP_AS_OWNER'],
        'opts': {
            'subnetlookup': True,
            'maxsubnet': 24,
        }
    }

2. Handle ASN Events and Emit Netblocks

def handleEvent(self, event):
    if event.eventType != 'BGP_AS_OWNER':
        return
    
    asn = event.data
    prefixes = self.query_asn_api(asn)  # Your data source

    
    for prefix in prefixes:
        # Register as alias of original target

        self.sf.target.setAlias(prefix, "NETBLOCK_OWNER")
        
        # Emit netblock event for downstream modules

        netblock_event = SpiderFootEvent(
            "NETBLOCK_OWNER",
            prefix,
            self.__name__,
            event
        )
        self.notifyListeners(netblock_event)

3. Enable Subnet Expansion (Optional)

If your module also enumerates individual IPs from netblocks, implement the standard size-checking pattern shown earlier.

Key Source Files Reference

File	Purpose
spiderfoot/target.py	SpiderFootTarget class; matches() and setAlias() methods
spiderfoot/db.py	Event type definitions for network objects
modules/sfp_ripe.py	Canonical ASN-to-netblock implementation using RIPE
modules/sfp_bgpview.py	Alternative BGP data source implementation
sf.py / sfscan.py	CLI entry points; target type parsing
spiderfoot/correlation.py	Subnet matching rules for relationship linking

Summary

• SpiderFoot treats subnets and ASNs as first-class targets through dedicated event types and the SpiderFootTarget class
• The matches() method in target.py provides subnet containment testing using netaddr
• The alias mechanism via setAlias() links discovered prefixes to original ASN targets
• Modules like sfp_ripe.py demonstrate the canonical pattern: query ASN API, register aliases, emit NETBLOCK_OWNER events
• Size limits through maxsubnet options prevent runaway enumeration on large allocations
• The correlation engine automatically links IPs to parent netblocks using match_method='subnet' rules

Frequently Asked Questions

What is the difference between NETBLOCK_OWNER and BGP_AS_OWNER in SpiderFoot?

BGP_AS_OWNER represents an Autonomous System Number registration—the organization that controls a range of IP addresses. NETBLOCK_OWNER represents the actual IP prefix or subnet allocated to that organization. When you scan an ASN, modules emit BGP_AS_OWNER for the ASN itself, then expand it to multiple NETBLOCK_OWNER events for each owned prefix.

How does SpiderFoot prevent scanning enormous subnets like /8 networks?

Each module that supports subnet enumeration includes a maxsubnet option (typically defaulting to /24). The code in sfp_virustotal.py and similar modules checks IPNetwork(eventData).prefixlen against this limit, skipping expansion for larger networks. Users can adjust this threshold per-module based on their scanning requirements.

Can I scan both an ASN and a specific subnet in the same SpiderFoot operation?

Yes. SpiderFoot supports multiple targets and target types simultaneously. You can invoke sf.py with multiple -s and -t pairs, or use the web interface to add targets. The alias mechanism ensures that subnets discovered from ASN expansion and explicitly targeted subnets are treated consistently throughout the correlation engine.

What external data sources does SpiderFoot use for ASN information?

The primary implementation is sfp_ripe.py, which queries the RIPE NCC Database API for ASN-to-prefix mappings. Alternative implementations include sfp_bgpview.py (using BGPView API) and sfp_hackertarget.py. Each follows the same pattern of ASN event handling and netblock emission, allowing you to select or combine sources based on data quality and rate limit requirements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}