How `request_payload` Works in Sherlock for POST-Based Sites: A Complete Guide

Sherlock's request_payload field lets you send JSON data in POST or PUT requests by interpolating the target username into a template defined in data.json, with the request logic automatically handled in sherlock_project/sherlock.py.

The Sherlock project is an open-source username enumeration tool that queries hundreds of social platforms. While most sites use simple GET requests, some APIs—particularly GraphQL endpoints and modern REST services—require POST requests with structured JSON bodies. The request_payload mechanism enables Sherlock to support these advanced authentication patterns without modifying the core Python code.

Understanding request_payload in Sherlock's Architecture

The request_payload field is an optional configuration property defined in the site manifest (data.json). When present, it instructs Sherlock to send an HTTP request with a JSON body rather than a query string.

Two fields work together to enable POST-based queries:

• request_method – Specifies the HTTP verb (e.g., "POST", "PUT"). Defaults to GET if omitted.
• request_payload – A JSON-serializable object (dict) containing the request body template.

This architecture allows Sherlock to query complex endpoints like GraphQL services where the username must be embedded inside a JSON query structure.

How request_payload Works Under the Hood

The implementation follows a four-step pipeline inside sherlock_project/sherlock.py. Understanding this flow helps when debugging custom site configurations or contributing new POST-based detectors.

Step 1: Defining the Payload in data.json

Site definitions reside in sherlock_project/resources/data.json. A POST-based entry includes the request_method and request_payload keys.

For example, the Anilist configuration (lines 99–105) uses a GraphQL query:

{
  "Anilist": {
    "errorType": "json",
    "errorMsg": "User not found",
    "request_method": "POST",
    "request_payload": {
      "query": "query($name:String){User(name:$name){id}}",
      "variables": {"name": "{}"}
    },
    "url": "https://graphql.anilist.co/",
    "urlMain": "https://anilist.co/",
    "username_claimed": "known_user"
  }
}

The {} placeholder indicates where the target username will be inserted.

Step 2: Interpolating the Username

Before sending the request, Sherlock processes the payload template to inject the actual username. In sherlock_project/sherlock.py (lines 264–283), the code retrieves the payload and applies string interpolation:

# Extract payload template from site configuration

request_payload = net_info.get("request_payload")
if request_payload:
    # Replace {} or {username} placeholders with target username

    request_payload = interpolate_string(request_payload, username)

The interpolate_string function recursively traverses the JSON object, replacing any string value containing {} with the username. This allows flexible placement of the username anywhere in the JSON structure—whether in a GraphQL variable, a REST API field, or a nested object.

Step 3: Executing the POST Request

The final stage constructs and dispatches the HTTP request. Lines 312–330 in sherlock_project/sherlock.py determine the correct request method and pass the interpolated payload:

# Determine request function based on request_method (default: GET)

request_method = net_info.get("request_method", "GET").lower()
if request_method == "post":
    request = session.post
elif request_method == "put":
    request = session.put
else:
    request = session.get

# Execute request with JSON payload if present

if request_payload:
    future = request(url, json=request_payload, ...)
else:
    future = request(url, ...)

By passing the payload to the json= parameter, the requests-future library automatically serializes the Python dictionary to JSON and sets the Content-Type: application/json header. This eliminates manual encoding and ensures compatibility with modern APIs.

Practical Example: Configuring a POST-Based Site

To add a new site requiring POST data, append an entry to sherlock_project/resources/data.json following this structure:

{
  "MyGraphQLService": {
    "errorType": "json",
    "errorMsg": "user not found",
    "request_method": "POST",
    "request_payload": {
      "query": "query { user(username: \"{}\") { id } }"
    },
    "url": "https://api.example.com/graphql",
    "urlMain": "https://example.com",
    "username_claimed": "existing_user"
  }
}

Key configuration points:

• request_method: Must be "POST" (or "PUT") to trigger the payload logic.
• request_payload: Any JSON-serializable structure. Use {} as the username placeholder.
• errorType: For JSON APIs, use "json" combined with errorMsg to detect non-existent users via response content.

When Sherlock processes this entry, it substitutes the target username into the query string and submits a POST request to the GraphQL endpoint.

Key Implementation Files

The request_payload functionality spans three core files in the sherlock-project/sherlock repository:

• sherlock_project/sherlock.py – Contains the request orchestration logic, including payload interpolation (lines 264–283) and the HTTP method dispatch (lines 312–330).
• sherlock_project/resources/data.json – The site manifest where request_method and request_payload are defined for POST-based targets like Anilist.
• sherlock_project/sites.py – Loads and validates the JSON manifest, exposing site configurations to the search engine.

Summary

• request_payload enables Sherlock to query APIs requiring POST or PUT requests with JSON bodies.
• The payload template uses {} placeholders that get interpolated with the target username before the request is sent.
• Sherlock automatically serializes the payload and sets Content-Type: application/json when passing data to the json= parameter of the request function.
• Configuration occurs entirely within data.json through the request_method and request_payload fields, requiring no changes to the core Python code.

Frequently Asked Questions

What is request_payload in Sherlock?

request_payload is a JSON field in Sherlock's site configuration (data.json) that defines the request body for sites requiring POST or PUT methods. It allows the tool to send structured data—such as GraphQL queries or API parameters—instead of simple GET requests. The payload supports username interpolation using {} placeholders.

How does Sherlock handle username interpolation in POST requests?

Sherlock extracts the request_payload dictionary from the site configuration and passes it through the interpolate_string function (found in sherlock_project/sherlock.py lines 264–283). This function recursively replaces any occurrence of {} or "{username}" with the actual target username throughout the JSON structure, ensuring the username appears correctly in nested GraphQL variables or API fields.

Can request_payload be used with HTTP methods other than POST?

Yes, while request_payload is most commonly used with POST requests, Sherlock supports any HTTP verb specified in the request_method field. If you set "request_method": "PUT", the tool will use session.put() instead of session.post(), but it will still pass the interpolated request_payload via the json= argument. The default method remains GET if request_method is omitted.

Where is the request_payload logic implemented in the Sherlock codebase?

The core logic resides in sherlock_project/sherlock.py. Lines 264–283 handle the extraction and interpolation of the payload, while lines 312–330 manage the request dispatch—selecting the appropriate HTTP method function and passing the payload to the json= parameter. The site definitions themselves are stored in sherlock_project/resources/data.json, where entries like Anilist (lines 99–105) demonstrate practical usage of POST payloads for GraphQL endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}