How Holehe Categorizes Target Websites into Modules: Dynamic Discovery Explained

Holehe organizes target websites into functional modules using a filesystem-based category structure under holehe.modules, with automatic discovery via pkgutil.walk_packages and no manual registration required.

The open-source OSINT tool Holehe (by megadose) checks whether an email address is registered on hundreds of websites. Its modular architecture lets developers add new services without touching core code. This article explains how Holehe categorizes target websites into modules based on the actual source implementation.

Filesystem-Based Module Categorization

Holehe uses Python package hierarchy as its categorization system. The holehe/modules/ directory contains sub-packages named after vertical categories, with each website implemented as a standalone Python file.

holehe/
 └─ modules/
     ├─ social_media/
     │   ├─ facebook.py
     │   ├─ twitter.py
     │   └─ ...
     ├─ music/
     │   ├─ spotify.py
     │   └─ ...
     ├─ transport/
     │   ├─ blablacar.py
     │   └─ ...
     ├─ shopping/
     ├─ crm/
     └─ ...

Each .py file exports one asynchronous function matching the filename. For example, facebook.py contains async def facebook(email, client, out).

Dynamic Module Discovery in holehe/core.py

The categorization system relies on two key functions in holehe/core.py that discover modules at runtime without explicit imports.

import_submodules Walks the Package Tree

# From holehe/core.py lines 37-47

def import_submodules(package: str) -> tuple:
    """Import all submodules of a package, recursively."""
    package = importlib.import_module(package)
    return {
        name: importlib.import_module(f"{package.__name__}.{name}")
        for _, name, _ in pkgutil.walk_packages(package.__path__)
    }

This uses pkgutil.walk_packages to traverse every subdirectory under holehe.modules and eagerly imports each module found.

get_functions Filters Valid Targets

# From holehe/core.py lines 50-63

def get_functions(modules: dict) -> list:
    """Extract callable functions from imported modules."""
    functions = []
    for module_path, module in modules.items():
        parts = module_path.split('.')
        # Only paths like: holehe.modules.<category>.<site>

        if len(parts) > 3:
            for name, obj in inspect.getmembers(module):
                if callable(obj) and name == parts[-1]:
                    functions.append(obj)
    return functions

The three-component path check (holehe.modules.<category>.<site>) enforces the categorization convention. Any module deeper than holehe.modules is treated as a target website.

How New Categories and Sites Are Added

The filesystem-based approach enables zero-registration extensibility:

1. Create a new folder under holehe/modules/ for a new category
2. Add a Python file with an async function matching the filename
3. Restart Holehe — the discovery logic automatically includes it

This design removes the need for:

• Central registries or configuration files
• Decorator-based registration
• Manual import statements in core code

Code Examples: Working with Categorized Modules

Running a Single Category Module

import trio
import httpx
from holehe.modules.social_media.facebook import facebook

async def check_facebook():
    email = "target@example.com"
    out = []
    async with httpx.AsyncClient() as client:
        await facebook(email, client, out)
    print(out)  # [{'name': 'facebook', 'exists': ..., ...}]

trio.run(check_facebook)

Running All Discovered Modules Concurrently

import trio
import httpx
from holehe.core import import_submodules, get_functions

async def run_full_probe(email: str):
    # Dynamically discover all categorized modules

    all_modules = import_submodules('holehe.modules')
    functions = get_functions(all_modules)
    
    out = []
    async with httpx.AsyncClient() as client:
        async with trio.open_nursery() as nursery:
            for func in functions:
                nursery.start_soon(func, email, client, out)
    
    return out

# Execute probe across all categories

results = trio.run(run_full_probe, "target@example.com")

Key Source Files for Module Categorization

File	Purpose
holehe/core.py	Implements import_submodules() and get_functions() for dynamic discovery
holehe/modules/__init__.py	Empty package marker; enables subpackage imports
holehe/modules/social_media/facebook.py	Example module showing required async signature
holehe/modules/ subdirectories	Category folders (music/, transport/, shopping/, etc.)

Summary

• Holehe categorizes target websites using Python package subdirectories as category folders
• Dynamic discovery via pkgutil.walk_packages in holehe/core.py eliminates manual registration
• Path depth validation (holehe.modules.<category>.<site>) determines valid target modules
• Zero-config extensibility — adding websites requires only new files, no core changes
• Uniform execution interface — every module exports one async function with identical signature

Frequently Asked Questions

How does Holehe know which modules to load without a central registry?

Holehe scans the holehe.modules package tree at runtime using pkgutil.walk_packages in import_submodules(). Any Python file found in a subfolder is automatically imported and evaluated by get_functions() if its import path has more than three components.

Can I add my own custom category folder?

Yes. Create any new directory under holehe/modules/ (for example, holehe/modules/gaming/). Place Python files inside with async functions matching their filenames. The discovery logic will include them automatically on the next run.

What happens if two modules have the same name in different categories?

The full dotted path distinguishes them. holehe.modules.social_media.twitter and holehe.modules.news.twitter would both be valid, though this naming collision is discouraged for clarity.

Does Holehe support nested subcategories beyond one level?

No. The get_functions() check requires exactly holehe.modules.<category>.<site> (four path components). Nested structures like holehe/modules/social_media/video/tiktok.py would be ignored by the current discovery logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}