# How to Use SpiderFoot Exclusively via the Command-Line Interface (CLI)

> Master SpiderFoot via the command-line interface (CLI). Control scans, retrieve data, and export results without the web UI. Learn how to use sf.py and sfcli.py for full automation.

- Repository: [Steve Micallef/spiderfoot](https://github.com/smicallef/spiderfoot)
- Tags: how-to-guide
- Published: 2026-08-15

---

**SpiderFoot can be operated entirely from the command line by running the server component ([`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py)) and interacting with it through the [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) client, which provides complete scan management, data retrieval, and export capabilities without requiring the web UI.**

The `smicallef/spiderfoot` project offers a full-featured **CLI client** that mirrors every function available in the browser interface. For security practitioners, threat intelligence teams, and automation engineers who prefer terminal-based workflows or need to script reconnaissance operations, this command-line approach eliminates browser overhead while maintaining full access to SpiderFoot's reconnaissance engine.

---

## Understanding the SpiderFoot CLI Architecture

SpiderFoot's architecture separates concerns between **server** and **client**:

| Component | File | Purpose |
|-----------|------|---------|
| Server | [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) | Runs the Flask-based scan engine and REST API |
| CLI Client | [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) | Interactive terminal interface built on Python's `cmd.Cmd` class |

The CLI communicates with the server via HTTP requests to the JSON API. All commands you issue in [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) are translated to REST calls through the `request()` helper method, with responses formatted for terminal display.

Configuration options—including server URL, authentication credentials, and SSL verification—are stored in the `ownopts` dictionary at lines 66-78 of [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py). You can modify these at startup or interactively using the `set` command.

---

## Starting the Server and Launching the CLI

Before using the CLI, you must start the SpiderFoot server. The server listens on `http://127.0.0.1:5001` by default.

```bash

# Launch the server in the background

python sf.py &

```

Once the server is running, start the interactive CLI:

```bash
python sfcli.py

```

You'll see a prompt (`>>>`) where you can enter commands. First, verify connectivity:

```python
>>> ping

```

---

## Essential SpiderFoot CLI Commands

The CLI implements all functionality through `do_<cmd>` methods. Here are the core commands for **SpiderFoot CLI usage**:

### Survey Available Resources

```python
>>> modules           # List all reconnaissance modules (sfp_dns, sfp_shodan, etc.)

>>> types             # Show all data element types (EMAILADDR, IP_ADDRESS, etc.)

```

### Execute Scans

Start a scan with the `start` command, specifying target, modules, and optional name:

```python
>>> start example.com -m sfp_dns,sfp_shodan -n "DNS and Shodan Recon"

```

The `do_start` method (lines 54-102 in [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py)) constructs the API request and returns the scan ID for tracking.

### Monitor and Retrieve Data

```python
>>> scans -x          # Extended list of all scans

>>> data 1 -t EMAILADDR -u   # Unique email addresses from scan ID 1

>>> search 1 -p "admin"      # Search scan results for pattern

>>> summary 1 -t      # Summary table of scan results by type

```

The `do_search` method (lines 138-147) and `do_export` method (lines 191-210) handle result filtering and format conversion.

### Export and Logging

```python
>>> export 1 -t csv -f results.csv   # Export scan 1 to CSV

>>> logs 1 -w                        # Stream live logs (Ctrl-C to stop)

>>> delete 1                         # Remove scan and associated data

```

---

## Non-Interactive and Scripted Usage

For automation, **SpiderFoot CLI automation** supports execution without an interactive session.

### Single Command Execution

Pipe commands directly to [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py):

```bash
echo "summary 2 -t" | python sfcli.py

```

### Batch Command Files

Create a command script and execute with `-e`:

```bash
cat > commands.txt << 'EOF'
set cli.server_baseurl http://remote-server:5001
set cli.username admin
set cli.password secret123
start 10.0.0.0/8 -u footprint -n "Internal Network Scan"
scans -x
logs 5 -w
EOF

python sfcli.py -e commands.txt

```

This pattern enables **headless SpiderFoot operation** in CI/CD pipelines, cron jobs, or orchestration workflows.

---

## Connecting to Remote SpiderFoot Servers

The CLI is not restricted to local servers. Configure remote access with the `set` command:

```python
>>> set cli.server_baseurl https://spiderfoot.company.internal:5001
>>> set cli.username scanner-user
>>> set cli.password api-key-here
>>> set cli.ssl_verify false    # For self-signed certificates (use cautiously)

```

All subsequent commands route to the specified remote instance, enabling **distributed reconnaissance operations** where a central server collects data from multiple CLI clients.

---

## Summary

- **SpiderFoot CLI operation** requires starting [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) (server) and connecting via [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) (client)
- The CLI implements all web UI functions through `do_<cmd>` methods that call the REST API
- Interactive mode provides exploratory reconnaissance with real-time feedback
- Non-interactive mode via pipe or `-e <file>` enables full automation
- Remote server configuration supports distributed and containerized deployments
- Key source files: [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) (CLI implementation), [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) (server), [`spiderfoot/event.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/event.py) (data types), [`spiderfoot/plugin.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/plugin.py) (module base)

---

## Frequently Asked Questions

### What Python version is required for SpiderFoot CLI usage?

SpiderFoot requires **Python 3.7 or newer**. Both [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) and [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py) use contemporary Python features including `asyncio` for the server and `cmd.Cmd` from the standard library for the CLI. Verify your version with `python --version` before installation.

### Can I run SpiderFoot CLI without starting the server?

No. The CLI client ([`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py)) is strictly a frontend—it **cannot operate standalone**. The [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) server must be running to process scan requests, module execution, and data storage. The CLI sends all commands via HTTP to the server's REST endpoints as defined in [`spiderfoot/db.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/db.py) and [`spiderfoot/__init__.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/__init__.py).

### How do I list all available modules from the command line?

Use the `modules` command in [`sfcli.py`](https://github.com/smicallef/spiderfoot/blob/main/sfcli.py). This queries the `/modules` endpoint, which enumerates all classes inheriting from `SpiderFootPlugin` in [`spiderfoot/plugin.py`](https://github.com/smicallef/spiderfoot/blob/main/spiderfoot/plugin.py). For scripting: `echo "modules" | python sfcli.py | grep sfp_dns` to filter for specific module types.

### Is authentication required for CLI access?

**Authentication depends on server configuration.** If [`sf.py`](https://github.com/smicallef/spiderfoot/blob/main/sf.py) was started with `--username` and `--password`, the CLI must provide matching credentials via `set cli.username` and `set cli.password`. Without server-side authentication, the CLI connects anonymously. Review your `~/.spiderfoot/spiderfoot.cfg` or startup flags to determine current security settings.