How Sherlock Handles Different HTTP Request Methods (GET, HEAD, POST, PUT)

Sherlock maps the optional request_method field in site definitions to specific requests_futures session methods, defaulting to GET when unspecified and raising a RuntimeError for unsupported verbs.

The sherlock-project/sherlock tool performs automated username enumeration across hundreds of social platforms. While most sites require simple GET requests, some endpoints demand specific HTTP verbs for proper probing. Understanding how Sherlock handles these different request methods reveals the flexibility of its networking layer implemented in sherlock_project/sherlock.py.

Understanding the Request Method Configuration

Each site definition in Sherlock uses a net-info dictionary that may contain a request_method key. This value dictates which HTTP verb Sherlock uses when contacting the target URL.

During execution, Sherlock extracts this configuration at line 263:

request_method = net_info.get("request_method")

If present, this string (typically "GET", "HEAD", "POST", or "PUT") determines the specific session method invoked. If absent, Sherlock silently defaults to standard GET behavior, which covers the majority of supported platforms.

Mapping HTTP Verbs to Session Methods

Sherlock utilizes a SherlockFuturesSession (lines 211-222), a threaded wrapper around the requests library that enables concurrent network operations. After parsing the request_method, Sherlock maps the string to the corresponding session callable:

GET Requests

GET serves as the default and most common method. When request_method equals "GET" or is omitted, Sherlock invokes:

request = session.get

This occurs at line 268 in sherlock_project/sherlock.py.

HEAD Requests

For endpoints where retrieving only headers suffices (reducing bandwidth), HEAD requests are supported:

request = session.head

This mapping happens at line 270, ideal for status-check endpoints that return 200 OK without body content.

POST Requests

Some platforms require POST requests for username validation or search functionality:

request = session.post

Line 272 handles this mapping, though POST usage in Sherlock typically requires additional configuration for request bodies in custom site definitions.

PUT Requests

Though less common for username enumeration, PUT support exists for specific API interactions:

request = session.put

This is implemented at line 274 in the main driver file.

Error Handling for Unsupported Methods

Sherlock enforces strict validation of HTTP verbs. If the request_method field contains anything other than the four supported strings, the application raises a clear runtime exception at line 277:

raise RuntimeError(f"Unsupported request_method for {url}")

This prevents silent failures and ensures that site configuration errors are caught immediately during the username search operation.

Practical Implementation Examples

When defining custom sites or modifying existing ones, you can specify the request method directly in the site configuration:

# Example: Adding a custom site requiring POST requests

custom_site = {
    "url": "https://api.example.com/user/{}",
    "request_method": "POST",          # Forces POST instead of default GET

    "regex": r"user_exists",
    "timeout": 10,
}

# Usage with Sherlock

from sherlock_project.sherlock import Sherlock
s = Sherlock(extra_sites=[custom_site])
s.run()

For command-line usage, modify the JSON site definition files to include the "request_method" key. Most built-in sites in sherlock_project/sites.py omit this field, defaulting to GET for maximum compatibility.

Summary

• Configuration: Sherlock reads the optional request_method field from each site's net-info dictionary in sherlock_project/sherlock.py (line 263).
• Method Mapping: The tool maps GET, HEAD, POST, and PUT strings to their corresponding SherlockFuturesSession methods (lines 268, 270, 272, 274).
• Default Behavior: When request_method is unspecified, Sherlock defaults to GET requests.
• Error Handling: Unsupported HTTP verbs trigger a RuntimeError with the target URL (line 277).
• Concurrency: All requests execute through a threaded FuturesSession wrapper enabling parallel username enumeration.

Frequently Asked Questions

What happens if request_method is not specified in a site definition?

Sherlock defaults to GET requests. The code at line 263 extracts the method using .get("request_method"), which returns None when the key is absent. The subsequent conditional logic treats None as a signal to use the default GET behavior, covering the vast majority of supported platforms.

Can Sherlock handle PATCH or DELETE requests?

No. Sherlock explicitly supports only GET, HEAD, POST, and PUT methods. If you configure a site with "request_method": "PATCH" or "DELETE", the application raises a RuntimeError at line 277 stating "Unsupported request_method for {url}". You would need to modify sherlock_project/sherlock.py to add support for additional verbs.

How does Sherlock execute requests concurrently?

Sherlock wraps the standard requests library in a SherlockFuturesSession (lines 211-222), which extends requests_futures.FuturesSession. This creates a thread pool that allows multiple HTTP requests (GET, HEAD, POST, or PUT) to execute in parallel. The selected request method (session.get, session.head, etc.) is invoked within this threaded context, significantly speeding up username enumeration across hundreds of sites.

Where are the HTTP methods defined for built-in sites?

Built-in site definitions reside in sherlock_project/sites.py. Most entries omit the request_method key, relying on the default GET behavior. However, you can override this by adding "request_method": "HEAD" (or POST/PUT) to specific site dictionaries in that file or via custom site configurations passed to the Sherlock class constructor.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}