How Sherlock Detects WAF Blocks (Cloudflare) and Returns QueryStatus.WAF

Sherlock detects Web Application Firewall blocks by scanning HTTP responses for hard-coded fingerprint strings in WAFHitMsgs and returns the QueryStatus.WAF status to indicate the request was intercepted.

When investigating username availability across social networks, the sherlock-project/sherlock tool must distinguish between actual profile absence and security interceptions. The codebase implements a specific Sherlock WAF detection mechanism that identifies when requests are blocked by protective services like Cloudflare, AWS CloudFront, or PerimeterX, returning a dedicated status code to signal this condition.

How Sherlock Detects WAF Blocks

The WAFHitMsgs Fingerprint Database

In sherlock_project/sherlock.py, Sherlock maintains a hard-coded list named WAFHitMsgs containing distinctive text fragments found exclusively on WAF challenge pages. This array includes CSS selectors and JavaScript markers from major protection services:

• Cloudflare loading spinner CSS: r'.loading-spinner{visibility:hidden}body.no-js .challenge-running{display:none}…'
• Cloudflare error text span: r'<span id="challenge-error-text">'
• AWS CloudFront integration token: r'AwsWafIntegration.forceRefreshToken'
• PerimeterX identifier object: r'{return l.onPageView}}),Object.defineProperty(r,"perimeterxIdentifiers",{enumerable:'

Response Content Scanning

After each asynchronous HTTP request completes, Sherlock examines the raw response text. The detection logic performs a substring search against the WAFHitMsgs patterns:

elif any(hitMsg in r.text for hitMsg in WAFHitMsgs):
    query_status = QueryStatus.WAF

This check occurs in the core request handling loop within sherlock_project/sherlock.py (lines 393-397), executing only after confirming no explicit error message was returned by the transport layer.

QueryStatus.WAF Return Value

When WAF fingerprints match, Sherlock sets the query result to QueryStatus.WAF, an enumeration member defined in sherlock_project/result.py:

class QueryStatus(Enum):
    ...
    WAF = "WAF"   # Request blocked by WAF (i.e. Cloudflare)

This constant provides a machine-readable signal that distinguishes security blocks from standard "username not found" or "username exists" states, enabling downstream logic to handle retries or proxy rotation appropriately.

User-Facing Output for WAF Blocks

The console notifier in sherlock_project/notify.py translates the QueryStatus.WAF status into human-readable warnings. When formatting results, the NotifyResult class emits a red status line indicating the interception:

elif result.status == QueryStatus.WAF:
    print("[ -] site_name: Blocked by bot detection (proxy may help)")

This output alerts users that the target site actively blocked the request and suggests using a proxy to bypass the restriction.

Practical Code Examples

Command Line WAF Detection

Running Sherlock against a protected site produces immediate visual feedback:

$ sherlock johndoe
[*] Checking site: Instagram
[-] Instagram: Blocked by bot detection (proxy may help)

The [-] prefix and "Blocked by bot detection" message confirm the tool identified a Cloudflare or similar WAF challenge page.

Programmatic WAF Handling in Python

When using Sherlock as a library, inspect the QueryStatus enumeration to handle WAF blocks programmatically:

from sherlock_project.sherlock import Sherlock
from sherlock_project.result import QueryStatus

sherlock = Sherlock(
    username="johndoe",
    sites=["instagram", "twitter"],
    timeout=30,
)

results = sherlock.run()

for res in results:
    if res.status == QueryStatus.WAF:
        print(f"{res.site_name} blocked by WAF - retry with proxy")
    elif res.status == QueryStatus.CLAIMED:
        print(f"{res.site_name} username exists")
    else:
        print(f"{res.site_name} status: {res.status}")

The res.status == QueryStatus.WAF condition captures exactly the scenario where the WAFHitMsgs fingerprint search matched the response content.

Summary

• Sherlock detects Cloudflare, AWS CloudFront, and PerimeterX blocks by matching response content against hard-coded fingerprints in the WAFHitMsgs list.
• Upon detection, the tool assigns QueryStatus.WAF from sherlock_project/result.py to the query result.
• Users see "Blocked by bot detection (proxy may help)" in console output via sherlock_project/notify.py.
• The WAF status enables both manual troubleshooting and automated retry logic with proxy rotation.

Frequently Asked Questions

What specific WAF services does Sherlock detect?

Sherlock recognizes fingerprints from Cloudflare (loading spinner CSS and challenge error text), AWS CloudFront (AwsWafIntegration.forceRefreshToken), and PerimeterX (JavaScript object definitions). The WAFHitMsgs array in sherlock_project/sherlock.py contains these specific string patterns.

How does Sherlock differentiate between a WAF block and a missing username?

Standard username absence triggers different status codes like QueryStatus.AVAILABLE or QueryStatus.CLAIMED based on HTTP status codes and response selectors. WAF detection occurs earlier in the logic chain through substring matching against WAFHitMsgs, taking precedence by setting QueryStatus.WAF before standard existence checks execute.

Can Sherlock automatically retry requests when a WAF block is detected?

The current implementation does not include automatic retry logic for WAF blocks. The tool returns QueryStatus.WAF and prints a console message suggesting proxy usage. Users must implement their own retry mechanisms or configure proxies upstream when using Sherlock programmatically.

Where is the WAF detection logic located in the source code?

The detection strings reside in sherlock_project/sherlock.py within the WAFHitMsgs list (around lines 385-390). The substring search executing this detection appears at lines 393-397 in the same file. The resulting status enum definition lives in sherlock_project/result.py, while the console output formatting appears in sherlock_project/notify.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}