# Sherlock Maximum Concurrent Worker Count: How the 20-Worker Limit Works

> Understand Sherlock's maximum concurrent worker count. Learn how the 20-worker limit works and its automatic scaling for efficient social network probing.

- Repository: [Sherlock/sherlock](https://github.com/sherlock-project/sherlock)
- Tags: internals
- Published: 2026-03-02

---

**Sherlock caps concurrent worker threads at 20 when probing social networks, automatically scaling down to match the number of target sites only when the site list contains fewer than 20 entries.**

The Sherlock username investigation tool parallelizes HTTP requests across hundreds of social networks using a thread-pooled `FuturesSession`. To prevent resource exhaustion and respect remote server rate limits, the project enforces a strict **maximum concurrent worker count** based on the size of the site list being queried.

## How the Worker Count is Determined

In [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py), the core logic calculates the thread pool size before initiating network probes. Around lines 213–218, the code evaluates the length of the `site_data` dictionary against a fixed threshold:

```python

# Limit number of workers to 20.

if len(site_data) >= 20:
    max_workers = 20
else:
    max_workers = len(site_data)

```

This conditional ensures that **max_workers** never exceeds 20. When investigating usernames across Sherlock's entire site database, the tool caps parallelism at 20 threads. If you filter the query to a handful of sites, the worker count scales down to match the actual workload, avoiding unnecessary thread overhead.

## Implementation Details

The worker initialization occurs within the `sherlock()` function in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py). After calculating `max_workers`, the function passes this value to a `FuturesSession` executor, creating a bounded thread pool that manages all HTTP requests concurrently.

Key implementation characteristics:
- **Hard ceiling**: 20 workers maximum regardless of site count
- **Dynamic scaling**: Uses `len(site_data)` when the site list is smaller than the cap
- **Thread pool type**: `FuturesSession` from the `requests-futures` library

## Verifying Worker Allocation in Practice

### Checking the Limit Programmatically

When invoking Sherlock via Python, you can verify the effective worker count by inspecting the site data length against the built-in logic:

```python
from sherlock_project.sites import SitesInformation

site_data = SitesInformation().sites
site_count = len(site_data)

# Replicates Sherlock's internal logic

max_workers = 20 if site_count >= 20 else site_count

print(f"Total sites loaded: {site_count}")
print(f"Max workers allocated: {max_workers}")

```

### Command-Line Behavior

When running Sherlock from the terminal, the same 20-worker limit applies automatically:

```bash
python -m sherlock_project exampleuser

```

The CLI entry point in [`sherlock_project/__main__.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/__main__.py) calls the `sherlock()` function, inheriting the worker cap defined in the core module. Whether probing 5 sites or 500, the concurrency never exceeds 20 parallel requests.

## Why 20 Workers?

The **20-worker limit** balances scanning speed against network reliability and server load. While modern systems could theoretically handle more simultaneous connections, this threshold prevents excessive memory consumption from thread overhead, avoids triggering rate limits or IP bans on target platforms, and prevents network congestion that could degrade individual request performance.

## Summary

- **Sherlock caps concurrent workers at 20** to balance performance and resource usage when querying social networks.
- The logic resides in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) at lines 213–218.
- Worker count equals `min(20, len(site_data))`, scaling down only for smaller site lists.
- Both Python API and CLI usage inherit this limit automatically through the `sherlock()` function.
- The limit is hardcoded and applies to all username probes regardless of configuration.

## Frequently Asked Questions

### What is the maximum number of concurrent workers in Sherlock?

The absolute maximum is **20 workers**. This hard limit is defined in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) and applies to all scanning operations, regardless of how many sites are being queried or whether you use the Python API or command-line interface.

### How does Sherlock determine the number of workers to use?

Sherlock compares the length of the `site_data` list against the threshold of 20. If the site count is 20 or greater, it uses exactly 20 workers. If fewer sites are specified, the worker count matches the site count exactly to avoid creating idle threads.

### Can I increase the 20-worker limit in Sherlock?

No, not without modifying the source code. The value `20` is hardcoded in the conditional at lines 213–218 of [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py). To increase concurrency, you would need to edit this specific file and change the literal value, though this is not recommended as it may trigger rate limiting or degrade performance on target sites.

### Where is the worker limit defined in the Sherlock source code?

The worker calculation logic is located in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) between lines 213 and 218. This section creates the `max_workers` variable that gets passed to the `FuturesSession` executor, effectively capping the thread pool size for all username probes.