How Holehe Parses and Extracts CSRF Tokens from Website Responses

Holehe extracts CSRF tokens by performing an initial GET request to the target login page, then parsing the response using string splits, regular expressions, BeautifulSoup queries, or cookie jar inspection to locate tokens embedded in HTML meta tags, hidden form inputs, JavaScript JSON payloads, or HTTP cookies.

Holehe is an open-source OSINT tool maintained by megadose that checks whether an email address is registered on various websites. To bypass CSRF protections during these checks, the tool must dynamically parse and extract tokens from each service's unique implementation. Understanding how holehe parses and extracts CSRF tokens reveals lightweight, service-specific parsing strategies that differ from heavyweight generic scrapers.

The Four Primary CSRF Token Locations in Holehe

Across the codebase, tokens consistently appear in one of four locations. Rather than implementing a universal parser, holehe uses targeted extraction logic tailored to each service's specific DOM structure or cookie configuration.

HTML Meta Tags and Hidden Inputs

Many services embed tokens directly in the HTML markup. In holehe/modules/social_media/strava.py at line 23, the token resides in a meta tag:

token = response.text.split('<meta name="csrf-token" content="')[1].split('"')[0]

Similarly, the BuyMeACoffee module in holehe/modules/crowfunding/buymeacoffee.py (line 27) uses BeautifulSoup to parse a hidden input field:

csrf_token = soup.find(attrs={'name': 'bmc_csrf_token'}).get("value")

JavaScript-Embedded JSON Payloads

Modern React and SPA frameworks often embed configuration objects within script tags. The Instagram module at holehe/modules/social_media/instagram.py (line 22) extracts the token from a JSON payload using string splitting:

token = freq.text.split('{"config":{"csrf_token":"')[1].split('"')[0]

The Facebook module follows an identical pattern in holehe/modules/social_media/facebook.py (line 27), targeting the same {"config":{"csrf_token":" delimiter structure.

HTTP Cookies

Some services transmit CSRF tokens via Set-Cookie headers rather than HTML. In holehe/modules/social_media/xing.py (line 29), the module reads directly from the cookie jar:

headers['x-csrf-token'] = response.cookies["xing_csrf_token"]

The Eventbrite implementation in holehe/modules/products/eventbrite.py (line 25) uses a similar approach:

csrf_token = req.cookies["csrftoken"]

Extraction Techniques Used in Holehe Modules

Holehe employs four distinct parsing strategies depending on the service's architecture and the complexity of its token obfuscation.

String Splitting for JSON and Meta Tags

For tokens embedded in predictable patterns, holehe uses Python's native split() method rather than importing heavy parsing libraries. This approach minimizes dependencies and execution time. The Instagram and Facebook modules both split on the literal string {"config":{"csrf_token":", then split again on the closing quote to isolate the value.

Regular Expression Matching

When token positions vary or include dynamic characters, holehe falls back to regex. The RocketReach module in holehe/modules/osint/rocketreach.py (line 24) demonstrates this technique:

token = re.search(r'name="csrfmiddlewaretoken" value="(.*)"', response.text).group(1)

This pattern handles the Django-style csrfmiddlewaretoken input field where exact positional splitting would be unreliable.

BeautifulSoup Parsing for DOM Elements

For well-formed HTML with distinct attributes, holehe uses BeautifulSoup to ensure robustness against minor markup changes. The BuyMeACoffee implementation queries by attribute name rather than positional index, making it more resilient to template updates.

Direct Cookie Jar Inspection

The simplest and most reliable method involves reading from the requests.Session cookie dictionary. Both Xing and Eventbrite modules access response.cookies by key name, eliminating the need for HTML parsing entirely and reducing the risk of extraction failures due to markup minification.

Step-by-Step CSRF Token Workflow

Every holehe module follows a consistent four-phase pattern for token handling. The general implementation structure appears as follows:

def login(self, session, email):
    # Phase 1: Retrieve the login page

    resp = session.get(self.login_url)
    
    # Phase 2: Extract token using service-specific logic

    if "csrf-token" in resp.text:
        token = resp.text.split('csrf-token" content="')[1].split('"')[0]
    else:
        token = resp.cookies.get('csrftoken')
    
    # Phase 3: Prepare authenticated request

    headers = {
        'X-CSRFToken': token,
        'Referer': self.login_url
    }
    data = {
        'email': email,
        'csrfmiddlewaretoken': token
    }
    
    # Phase 4: Execute authentication request

    result = session.post(self.auth_url, data=data, headers=headers)
    return result.status_code == 200

This pattern prioritizes minimal HTTP overhead and avoids rendering JavaScript, making holehe significantly faster than headless browser alternatives while maintaining compatibility with standard CSRF protection mechanisms.

Summary

• Service-specific parsers: Holehe uses lightweight, targeted extraction methods rather than generic HTML parsers to minimize dependencies and maximize speed.
• Four extraction targets: Tokens are sourced from HTML meta tags (Strava), hidden inputs (BuyMeACoffee, RocketReach), JSON embeddings (Instagram, Facebook), or HTTP cookies (Xing, Eventbrite).
• Parsing techniques: The codebase employs string splitting for fixed patterns, regex for variable positions, BeautifulSoup for DOM reliability, and direct cookie inspection when available.
• Token reuse: Extracted values are immediately injected into headers (typically X-CSRFToken or x-csrf-token) or POST data (csrfmiddlewaretoken) for the subsequent authentication request.
• Implementation files: Key examples reside in holehe/modules/social_media/instagram.py, holehe/modules/social_media/xing.py, holehe/modules/crowfunding/buymeacoffee.py, and holehe/modules/osint/rocketreach.py.

Frequently Asked Questions

What is the most common method holehe uses to extract CSRF tokens?

String splitting is the predominant technique across the holehe codebase. Modules targeting Instagram, Facebook, and Strava use response.text.split() on predictable delimiters because it executes faster than DOM parsing and requires no external dependencies beyond the standard library.

Why does holehe use service-specific parsers instead of a generic extractor?

Each service implements CSRF protection differently—some use meta tags, others embed tokens in JavaScript variables, and many store them in cookies. A generic extractor would need complex conditional logic to handle every edge case, increasing maintenance overhead and failure rates. Service-specific parsers allow contributors to implement minimal, reliable extraction logic tailored to each site's unique markup structure.

How does holehe handle CSRF tokens stored in cookies?

When tokens reside in cookies, holehe reads directly from the requests session cookie jar. For example, the Xing module accesses response.cookies["xing_csrf_token"] and immediately assigns it to the x-csrf-token header. This method bypasses HTML parsing entirely and is less prone to breakage during site redesigns.

Can holehe's CSRF extraction methods be used for other web scraping tasks?

Yes, the patterns demonstrated in holehe—specifically the combination of initial GET requests, targeted string splitting or regex extraction, and header injection—are applicable to any Python-based web automation requiring CSRF token handling. The techniques work particularly well for APIs and login endpoints that do not require JavaScript execution to generate tokens.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}