# How `request_payload` Works in Sherlock for POST-Based Sites: A Complete Guide

> Understand how Sherlock's request_payload enables POST requests for sites. Learn to send JSON data by interpolating usernames for automated requests.

- Repository: [Sherlock/sherlock](https://github.com/sherlock-project/sherlock)
- Tags: deep-dive
- Published: 2026-03-02

---

**Sherlock's `request_payload` field lets you send JSON data in POST or PUT requests by interpolating the target username into a template defined in [`data.json`](https://github.com/sherlock-project/sherlock/blob/main/data.json), with the request logic automatically handled in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py).**

The Sherlock project is an open-source username enumeration tool that queries hundreds of social platforms. While most sites use simple GET requests, some APIs—particularly GraphQL endpoints and modern REST services—require POST requests with structured JSON bodies. The `request_payload` mechanism enables Sherlock to support these advanced authentication patterns without modifying the core Python code.

## Understanding `request_payload` in Sherlock's Architecture

The `request_payload` field is an optional configuration property defined in the site manifest ([`data.json`](https://github.com/sherlock-project/sherlock/blob/main/data.json)). When present, it instructs Sherlock to send an HTTP request with a JSON body rather than a query string.

Two fields work together to enable POST-based queries:

- **`request_method`** – Specifies the HTTP verb (e.g., `"POST"`, `"PUT"`). Defaults to GET if omitted.
- **`request_payload`** – A JSON-serializable object (dict) containing the request body template.

This architecture allows Sherlock to query complex endpoints like GraphQL services where the username must be embedded inside a JSON query structure.

## How `request_payload` Works Under the Hood

The implementation follows a four-step pipeline inside [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py). Understanding this flow helps when debugging custom site configurations or contributing new POST-based detectors.

### Step 1: Defining the Payload in [`data.json`](https://github.com/sherlock-project/sherlock/blob/main/data.json)

Site definitions reside in [`sherlock_project/resources/data.json`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/resources/data.json). A POST-based entry includes the `request_method` and `request_payload` keys.

For example, the Anilist configuration (lines 99–105) uses a GraphQL query:

```json
{
  "Anilist": {
    "errorType": "json",
    "errorMsg": "User not found",
    "request_method": "POST",
    "request_payload": {
      "query": "query($name:String){User(name:$name){id}}",
      "variables": {"name": "{}"}
    },
    "url": "https://graphql.anilist.co/",
    "urlMain": "https://anilist.co/",
    "username_claimed": "known_user"
  }
}

```

The `{}` placeholder indicates where the target username will be inserted.

### Step 2: Interpolating the Username

Before sending the request, Sherlock processes the payload template to inject the actual username. In [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) (lines 264–283), the code retrieves the payload and applies string interpolation:

```python

# Extract payload template from site configuration

request_payload = net_info.get("request_payload")
if request_payload:
    # Replace {} or {username} placeholders with target username

    request_payload = interpolate_string(request_payload, username)

```

The `interpolate_string` function recursively traverses the JSON object, replacing any string value containing `{}` with the username. This allows flexible placement of the username anywhere in the JSON structure—whether in a GraphQL variable, a REST API field, or a nested object.

### Step 3: Executing the POST Request

The final stage constructs and dispatches the HTTP request. Lines 312–330 in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) determine the correct request method and pass the interpolated payload:

```python

# Determine request function based on request_method (default: GET)

request_method = net_info.get("request_method", "GET").lower()
if request_method == "post":
    request = session.post
elif request_method == "put":
    request = session.put
else:
    request = session.get

# Execute request with JSON payload if present

if request_payload:
    future = request(url, json=request_payload, ...)
else:
    future = request(url, ...)

```

By passing the payload to the `json=` parameter, the `requests-future` library automatically serializes the Python dictionary to JSON and sets the `Content-Type: application/json` header. This eliminates manual encoding and ensures compatibility with modern APIs.

## Practical Example: Configuring a POST-Based Site

To add a new site requiring POST data, append an entry to [`sherlock_project/resources/data.json`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/resources/data.json) following this structure:

```json
{
  "MyGraphQLService": {
    "errorType": "json",
    "errorMsg": "user not found",
    "request_method": "POST",
    "request_payload": {
      "query": "query { user(username: \"{}\") { id } }"
    },
    "url": "https://api.example.com/graphql",
    "urlMain": "https://example.com",
    "username_claimed": "existing_user"
  }
}

```

**Key configuration points:**

- **`request_method`**: Must be `"POST"` (or `"PUT"`) to trigger the payload logic.
- **`request_payload`**: Any JSON-serializable structure. Use `{}` as the username placeholder.
- **`errorType`**: For JSON APIs, use `"json"` combined with `errorMsg` to detect non-existent users via response content.

When Sherlock processes this entry, it substitutes the target username into the query string and submits a POST request to the GraphQL endpoint.

## Key Implementation Files

The `request_payload` functionality spans three core files in the `sherlock-project/sherlock` repository:

- **[`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py)** – Contains the request orchestration logic, including payload interpolation (lines 264–283) and the HTTP method dispatch (lines 312–330).
- **[`sherlock_project/resources/data.json`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/resources/data.json)** – The site manifest where `request_method` and `request_payload` are defined for POST-based targets like Anilist.
- **[`sherlock_project/sites.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sites.py)** – Loads and validates the JSON manifest, exposing site configurations to the search engine.

## Summary

- **`request_payload`** enables Sherlock to query APIs requiring POST or PUT requests with JSON bodies.
- The payload template uses `{}` placeholders that get interpolated with the target username before the request is sent.
- Sherlock automatically serializes the payload and sets `Content-Type: application/json` when passing data to the `json=` parameter of the request function.
- Configuration occurs entirely within [`data.json`](https://github.com/sherlock-project/sherlock/blob/main/data.json) through the `request_method` and `request_payload` fields, requiring no changes to the core Python code.

## Frequently Asked Questions

### What is `request_payload` in Sherlock?

`request_payload` is a JSON field in Sherlock's site configuration ([`data.json`](https://github.com/sherlock-project/sherlock/blob/main/data.json)) that defines the request body for sites requiring POST or PUT methods. It allows the tool to send structured data—such as GraphQL queries or API parameters—instead of simple GET requests. The payload supports username interpolation using `{}` placeholders.

### How does Sherlock handle username interpolation in POST requests?

Sherlock extracts the `request_payload` dictionary from the site configuration and passes it through the `interpolate_string` function (found in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py) lines 264–283). This function recursively replaces any occurrence of `{}` or `"{username}"` with the actual target username throughout the JSON structure, ensuring the username appears correctly in nested GraphQL variables or API fields.

### Can `request_payload` be used with HTTP methods other than POST?

Yes, while `request_payload` is most commonly used with POST requests, Sherlock supports any HTTP verb specified in the `request_method` field. If you set `"request_method": "PUT"`, the tool will use `session.put()` instead of `session.post()`, but it will still pass the interpolated `request_payload` via the `json=` argument. The default method remains GET if `request_method` is omitted.

### Where is the `request_payload` logic implemented in the Sherlock codebase?

The core logic resides in [`sherlock_project/sherlock.py`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/sherlock.py). Lines 264–283 handle the extraction and interpolation of the payload, while lines 312–330 manage the request dispatch—selecting the appropriate HTTP method function and passing the payload to the `json=` parameter. The site definitions themselves are stored in [`sherlock_project/resources/data.json`](https://github.com/sherlock-project/sherlock/blob/main/sherlock_project/resources/data.json), where entries like Anilist (lines 99–105) demonstrate practical usage of POST payloads for GraphQL endpoints.