Sherlock Maximum Concurrent Worker Count: How the 20-Worker Limit Works

Sherlock caps concurrent worker threads at 20 when probing social networks, automatically scaling down to match the number of target sites only when the site list contains fewer than 20 entries.

The Sherlock username investigation tool parallelizes HTTP requests across hundreds of social networks using a thread-pooled FuturesSession. To prevent resource exhaustion and respect remote server rate limits, the project enforces a strict maximum concurrent worker count based on the size of the site list being queried.

How the Worker Count is Determined

In sherlock_project/sherlock.py, the core logic calculates the thread pool size before initiating network probes. Around lines 213–218, the code evaluates the length of the site_data dictionary against a fixed threshold:

# Limit number of workers to 20.

if len(site_data) >= 20:
    max_workers = 20
else:
    max_workers = len(site_data)

This conditional ensures that max_workers never exceeds 20. When investigating usernames across Sherlock's entire site database, the tool caps parallelism at 20 threads. If you filter the query to a handful of sites, the worker count scales down to match the actual workload, avoiding unnecessary thread overhead.

Implementation Details

The worker initialization occurs within the sherlock() function in sherlock_project/sherlock.py. After calculating max_workers, the function passes this value to a FuturesSession executor, creating a bounded thread pool that manages all HTTP requests concurrently.

Key implementation characteristics:

• Hard ceiling: 20 workers maximum regardless of site count
• Dynamic scaling: Uses len(site_data) when the site list is smaller than the cap
• Thread pool type: FuturesSession from the requests-futures library

Verifying Worker Allocation in Practice

Checking the Limit Programmatically

When invoking Sherlock via Python, you can verify the effective worker count by inspecting the site data length against the built-in logic:

from sherlock_project.sites import SitesInformation

site_data = SitesInformation().sites
site_count = len(site_data)

# Replicates Sherlock's internal logic

max_workers = 20 if site_count >= 20 else site_count

print(f"Total sites loaded: {site_count}")
print(f"Max workers allocated: {max_workers}")

Command-Line Behavior

When running Sherlock from the terminal, the same 20-worker limit applies automatically:

python -m sherlock_project exampleuser

The CLI entry point in sherlock_project/__main__.py calls the sherlock() function, inheriting the worker cap defined in the core module. Whether probing 5 sites or 500, the concurrency never exceeds 20 parallel requests.

Why 20 Workers?

The 20-worker limit balances scanning speed against network reliability and server load. While modern systems could theoretically handle more simultaneous connections, this threshold prevents excessive memory consumption from thread overhead, avoids triggering rate limits or IP bans on target platforms, and prevents network congestion that could degrade individual request performance.

Summary

• Sherlock caps concurrent workers at 20 to balance performance and resource usage when querying social networks.
• The logic resides in sherlock_project/sherlock.py at lines 213–218.
• Worker count equals min(20, len(site_data)), scaling down only for smaller site lists.
• Both Python API and CLI usage inherit this limit automatically through the sherlock() function.
• The limit is hardcoded and applies to all username probes regardless of configuration.

Frequently Asked Questions

What is the maximum number of concurrent workers in Sherlock?

The absolute maximum is 20 workers. This hard limit is defined in sherlock_project/sherlock.py and applies to all scanning operations, regardless of how many sites are being queried or whether you use the Python API or command-line interface.

How does Sherlock determine the number of workers to use?

Sherlock compares the length of the site_data list against the threshold of 20. If the site count is 20 or greater, it uses exactly 20 workers. If fewer sites are specified, the worker count matches the site count exactly to avoid creating idle threads.

Can I increase the 20-worker limit in Sherlock?

No, not without modifying the source code. The value 20 is hardcoded in the conditional at lines 213–218 of sherlock_project/sherlock.py. To increase concurrency, you would need to edit this specific file and change the literal value, though this is not recommended as it may trigger rate limiting or degrade performance on target sites.

Where is the worker limit defined in the Sherlock source code?

The worker calculation logic is located in sherlock_project/sherlock.py between lines 213 and 218. This section creates the max_workers variable that gets passed to the FuturesSession executor, effectively capping the thread pool size for all username probes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}