How to Add New Website Modules to Holehe's Modular Structure: A Step-by-Step Guide

Holehe automatically discovers and executes any new website-checking module placed under holehe/modules/ without requiring manual registration or configuration changes.

Holehe's architecture relies on dynamic module discovery to check whether an email address is registered across hundreds of websites. Understanding how this modular system works is essential for contributors who want to extend the tool's coverage. This guide explains the exact mechanism and provides a working template for adding new website modules to megadose/holehe.

How Holehe Discovers Modules Automatically

The core discovery system operates through three coordinated steps in holehe/core.py. No manual registry or configuration file maintenance is required.

Recursive Import of Sub-Packages

The import_submodules() function walks the entire package tree under holehe/modules and imports every .py file it encounters:

# holehe/core.py lines 37-47

def import_submodules(package, recursive=True):
    """Import all submodules of a module, recursively."""
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for _loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = package.__name__ + '.' + name
        results[full_name] = importlib.import_module(full_name)
        if recursive and is_pkg:
            results.update(import_submodules(full_name))
    return results

This recursive walk means you can organize modules in nested directories (social_media/, shopping/, developer_tools/) without modifying any import statements.

Extraction of Callable Functions

After import, get_functions() filters objects to identify valid site-checking functions:

# holehe/core.py lines 50-63

def get_functions(modules):
    """Extract all callable functions from imported modules."""
    websites = []
    for module in modules:
        try:
            if len(module.split(".")) > 3:  # e.g., holehe.modules.social_media.twitter

                exec(f"from {module} import *")
                for _, obj in globals().items():
                    if callable(obj) and obj.__module__ == module:
                        websites.append(obj)
        except Exception:
            pass
    return websites

The key filter len(module.split(".")) > 3 ensures only leaf-node modules (actual site implementations) are collected, excluding package-level utilities.

Async Execution via Trio Nursery

Each discovered function is scheduled in an async nursery with standardized parameters:

# holehe/core.py lines 19-21 (launch_module signature)

async def launch_module(function, email, client, out):
    # function: the site-checking coroutine

    # email: target email string

    # client: shared httpx.AsyncClient instance

    # out: list collecting all results

Creating a New Website Module: Complete Template

To add support for a new website, create a Python file in the appropriate category directory under holehe/modules/. The function name must match the filename exactly.

File Location and Naming Convention

Place your module in the logical category:

holehe/modules/
├── social_media/          # Facebook, Twitter, Instagram, etc.

├── shopping/             # Amazon, eBay, Etsy, etc.

├── developer_tools/      # GitHub, GitLab, Stack Overflow, etc.

├── [your_category]/      # Create if no existing category fits

└── __init__.py           # Required package marker (can be empty)

Minimal Working Module Template

# file: holehe/modules/shopping/newsite.py

import httpx


async def newsite(email: str, client: httpx.AsyncClient, out: list):
    """
    Check if an email is registered on newsite.com.

    Parameters
    ----------
    email : str
        The target email address to verify.
    client : httpx.AsyncClient
        Shared async HTTP client (reuse for connection pooling).
    out : list
        Append result dictionary to this shared list.

    Returns
    -------
    None (results appended to out)
    """
    # Use the shared client for all HTTP requests

    try:
        response = await client.get(
            f"https://api.newsite.com/v1/users/exists?email={email}",
            timeout=10
        )
        
        # Parse API response to determine account existence

        exists = response.status_code == 200 and response.json().get("exists")
        
    except httpx.RequestError:
        exists = False
        error = True
    else:
        error = False

    # Required result format (all fields must be present)

    result = {
        "name": "newsite",              # Internal identifier

        "domain": "newsite.com",        # Website domain

        "rateLimit": False,             # True if rate-limited

        "error": error,                 # True if request failed

        "exists": exists,               # True if email registered

        "emailrecovery": None,          # Partial recovery email if exposed

        "phoneNumber": None,            # Partial phone if exposed

        "others": None,                 # Any additional metadata

    }
    
    out.append(result)

Critical Implementation Requirements

• Function name matches filename: newsite.py must contain async def newsite(...).
• Async function: All modules must be declared with async def.
• Exact parameter signature: (email: str, client: httpx.AsyncClient, out: list).
• Complete result dictionary: All eight keys must be present with appropriate values.
• Shared client usage: Always use the provided client parameter—never create your own httpx.AsyncClient.

Verifying Your New Module Works

After creating your module, verify integration without waiting for a full scan:

# Run Holehe against a test email

$ holehe test@example.com

# The output will include your new module:

# [+] newsite.com

# [-] facebook.com

# ...

You can also test your module in isolation:

import asyncio
import httpx
from holehe.modules.shopping.newsite import newsite


async def test():
    async with httpx.AsyncClient() as client:
        results = []
        await newsite("test@example.com", client, results)
        print(results[0])


asyncio.run(test())

Reference: Key Source Files

File	Purpose	Location
holehe/core.py	Dynamic loader (import_submodules, get_functions, launch_module)	core.py
holehe/modules/__init__.py	Package marker (empty file)	modules/init.py
Example implementation	Production module showing error handling, rate limiting, response parsing	holehe/modules/social_media/twitter.py

Summary

• No registration required: Place any .py file under holehe/modules/ and it is automatically discovered.
• Name matching: The function name must exactly match the filename.
• Standard signature: Use async def name(email, client, out) with proper type hints.
• Required output: Return a dictionary with all eight fields: name, domain, rateLimit, error, exists, emailrecovery, phoneNumber, others.
• Shared resources: Reuse the provided httpx.AsyncClient and append results to the shared out list.
• Recursive organization: Nest modules in category directories as needed; the discovery system traverses all subdirectories.

Frequently Asked Questions

What happens if my function name doesn't match the filename?

Holehe will still import the module, but get_functions() may fail to extract your function or it will not be callable by its expected name. The dynamic loader relies on naming conventions to identify valid site-checking routines. Always ensure def mysite(): lives in mysite.py.

Can I use synchronous HTTP libraries like requests?

No. All modules run inside a Trio async nursery and must use async/await syntax. The provided httpx.AsyncClient is pre-configured for concurrent execution. Using blocking libraries like requests would halt the entire scan and likely trigger timeout errors.

Do I need to modify any configuration files when adding a module?

No configuration changes are necessary. The import_submodules() function in holehe/core.py recursively discovers all Python files at runtime. Simply place your module in the correct directory and restart Holehe.

How do I handle websites that require JavaScript rendering or browser automation?

Holehe's standard modules use direct HTTP requests via httpx. For JavaScript-heavy sites, you would need to implement Playwright or Selenium within your module, though this is discouraged for performance reasons. Consider whether the site exposes an internal API that returns JSON data without requiring browser execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}