How to Use SpiderFoot Exclusively via the Command-Line Interface (CLI)

SpiderFoot can be operated entirely from the command line by running the server component (sf.py) and interacting with it through the sfcli.py client, which provides complete scan management, data retrieval, and export capabilities without requiring the web UI.

The smicallef/spiderfoot project offers a full-featured CLI client that mirrors every function available in the browser interface. For security practitioners, threat intelligence teams, and automation engineers who prefer terminal-based workflows or need to script reconnaissance operations, this command-line approach eliminates browser overhead while maintaining full access to SpiderFoot's reconnaissance engine.

---

Understanding the SpiderFoot CLI Architecture

SpiderFoot's architecture separates concerns between server and client:

Component	File	Purpose
Server	sf.py	Runs the Flask-based scan engine and REST API
CLI Client	sfcli.py	Interactive terminal interface built on Python's cmd.Cmd class

The CLI communicates with the server via HTTP requests to the JSON API. All commands you issue in sfcli.py are translated to REST calls through the request() helper method, with responses formatted for terminal display.

Configuration options—including server URL, authentication credentials, and SSL verification—are stored in the ownopts dictionary at lines 66-78 of sfcli.py. You can modify these at startup or interactively using the set command.

---

Starting the Server and Launching the CLI

Before using the CLI, you must start the SpiderFoot server. The server listens on http://127.0.0.1:5001 by default.

# Launch the server in the background

python sf.py &

Once the server is running, start the interactive CLI:

python sfcli.py

You'll see a prompt (>>>) where you can enter commands. First, verify connectivity:

>>> ping

---

Essential SpiderFoot CLI Commands

The CLI implements all functionality through do_<cmd> methods. Here are the core commands for SpiderFoot CLI usage:

Survey Available Resources

>>> modules           # List all reconnaissance modules (sfp_dns, sfp_shodan, etc.)

>>> types             # Show all data element types (EMAILADDR, IP_ADDRESS, etc.)

Execute Scans

Start a scan with the start command, specifying target, modules, and optional name:

>>> start example.com -m sfp_dns,sfp_shodan -n "DNS and Shodan Recon"

The do_start method (lines 54-102 in sfcli.py) constructs the API request and returns the scan ID for tracking.

Monitor and Retrieve Data

>>> scans -x          # Extended list of all scans

>>> data 1 -t EMAILADDR -u   # Unique email addresses from scan ID 1

>>> search 1 -p "admin"      # Search scan results for pattern

>>> summary 1 -t      # Summary table of scan results by type

The do_search method (lines 138-147) and do_export method (lines 191-210) handle result filtering and format conversion.

Export and Logging

>>> export 1 -t csv -f results.csv   # Export scan 1 to CSV

>>> logs 1 -w                        # Stream live logs (Ctrl-C to stop)

>>> delete 1                         # Remove scan and associated data

---

Non-Interactive and Scripted Usage

For automation, SpiderFoot CLI automation supports execution without an interactive session.

Single Command Execution

Pipe commands directly to sfcli.py:

echo "summary 2 -t" | python sfcli.py

Batch Command Files

Create a command script and execute with -e:

cat > commands.txt << 'EOF'
set cli.server_baseurl http://remote-server:5001
set cli.username admin
set cli.password secret123
start 10.0.0.0/8 -u footprint -n "Internal Network Scan"
scans -x
logs 5 -w
EOF

python sfcli.py -e commands.txt

This pattern enables headless SpiderFoot operation in CI/CD pipelines, cron jobs, or orchestration workflows.

---

Connecting to Remote SpiderFoot Servers

The CLI is not restricted to local servers. Configure remote access with the set command:

>>> set cli.server_baseurl https://spiderfoot.company.internal:5001
>>> set cli.username scanner-user
>>> set cli.password api-key-here
>>> set cli.ssl_verify false    # For self-signed certificates (use cautiously)

All subsequent commands route to the specified remote instance, enabling distributed reconnaissance operations where a central server collects data from multiple CLI clients.

---

Summary

• SpiderFoot CLI operation requires starting sf.py (server) and connecting via sfcli.py (client)
• The CLI implements all web UI functions through do_<cmd> methods that call the REST API
• Interactive mode provides exploratory reconnaissance with real-time feedback
• Non-interactive mode via pipe or -e <file> enables full automation
• Remote server configuration supports distributed and containerized deployments
• Key source files: sfcli.py (CLI implementation), sf.py (server), spiderfoot/event.py (data types), spiderfoot/plugin.py (module base)

---

Frequently Asked Questions

What Python version is required for SpiderFoot CLI usage?

SpiderFoot requires Python 3.7 or newer. Both sf.py and sfcli.py use contemporary Python features including asyncio for the server and cmd.Cmd from the standard library for the CLI. Verify your version with python --version before installation.

Can I run SpiderFoot CLI without starting the server?

No. The CLI client (sfcli.py) is strictly a frontend—it cannot operate standalone. The sf.py server must be running to process scan requests, module execution, and data storage. The CLI sends all commands via HTTP to the server's REST endpoints as defined in spiderfoot/db.py and spiderfoot/__init__.py.

How do I list all available modules from the command line?

Use the modules command in sfcli.py. This queries the /modules endpoint, which enumerates all classes inheriting from SpiderFootPlugin in spiderfoot/plugin.py. For scripting: echo "modules" | python sfcli.py | grep sfp_dns to filter for specific module types.

Is authentication required for CLI access?

Authentication depends on server configuration. If sf.py was started with --username and --password, the CLI must provide matching credentials via set cli.username and set cli.password. Without server-side authentication, the CLI connects anonymously. Review your ~/.spiderfoot/spiderfoot.cfg or startup flags to determine current security settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}