How Holehe Dynamically Imports and Registers Modules at Runtime

Holehe discovers and executes service-check modules at runtime using pkgutil.walk_packages and importlib, eliminating the need for manual registration or hard-coded imports.

The open-source OSINT tool Holehe (maintained at megadose/holehe) probes hundreds of services for email account existence. Its architecture relies on dynamic module import and registration to stay extensible: developers drop a Python file into the modules directory, and the framework automatically discovers, loads, and schedules it for execution without modifying core code.

Runtime Module Discovery Architecture

The dynamic loading system in holehe/core.py operates through three coordinated phases: package tree traversal, callable extraction, and concurrent execution.

Recursive Package Walking with import_submodules

At lines 37–47 of holehe/core.py, the import_submodules function uses pkgutil.walk_packages to recursively traverse the holehe.modules package tree. For every submodule encountered, it constructs the full dotted name and imports it via importlib.import_module.

import importlib
import pkgutil

def import_submodules(package_name: str):
    """Recursively import every submodule of *package_name*."""
    package = importlib.import_module(package_name)
    loaded = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = f"{package.__name__}.{name}"
        loaded[full_name] = importlib.import_module(full_name)
        if is_pkg:
            loaded.update(import_submodules(full_name))
    return loaded

If the discovered entry is itself a package, the function recurses to ensure nested modules are also imported and added to the returned dictionary.

Extracting Callables via get_functions

Once import_submodules returns a dictionary mapping dotted names to module objects, get_functions (lines 50–63) filters this collection to isolate leaf modules—those whose dotted names contain more than three components (e.g., holehe.modules.social_media.instagram). It then extracts the public function from each module’s __dict__ using the final segment of the dotted name.

def get_functions(modules_dict):
    """Extract the public function from each leaf module."""
    functions = []
    for dotted_name, module in modules_dict.items():
        # Leaf modules have at least 4 parts: holehe.modules.<category>.<service>

        if len(dotted_name.split(".")) > 3:
            func_name = dotted_name.split(".")[-1]         # e.g., "instagram"

            functions.append(module.__dict__[func_name])     # The function object

    return functions

This convention requires that the function name inside instagram.py must exactly match the filename (def instagram(...)).

Concurrent Execution in maincore

The maincore function (lines 20–30) receives the list of callable functions and executes them concurrently using Trio’s nursery pattern. Each check receives the target email, a shared httpx.AsyncClient instance, and a results list to populate.

import trio
import httpx

async def launch_module(func, email, client, out):
    await func(email, client, out)

async def maincore(email):
    client = httpx.AsyncClient()
    out = []
    modules = import_submodules("holehe.modules")
    checks = get_functions(modules)
    
    async with trio.open_nursery() as nursery:
        for check in checks:
            nursery.start_soon(launch_module, check, email, client, out)
    
    await client.aclose()
    return out

Extending Holehe with New Services

Adding a new service check requires no changes to holehe/core.py. Create a Python file under holehe/modules/<category>/ where the filename matches the function name defined inside. For example, holehe/modules/social_media/instagram.py must contain def instagram(email, client, out):. The runtime import machinery automatically discovers the file on the next execution, extracts the callable, and schedules it for concurrent probing.

Summary

• import_submodules in holehe/core.py (lines 37–47) uses pkgutil.walk_packages and importlib.import_module to recursively load all submodules under holehe.modules.
• get_functions (lines 50–63) filters for leaf modules (dotted names with more than three segments) and extracts functions whose names match their filenames.
• maincore executes extracted callables concurrently via Trio’s nursery.start_soon(), passing shared HTTP clients and result collectors.
• The architecture supports drop-in extensions: adding a new service only requires placing a correctly named Python file in the appropriate subdirectory.

Frequently Asked Questions

How does Holehe find new modules without explicit imports?

Holehe traverses the holehe/modules directory at runtime using pkgutil.walk_packages, which yields every submodule regardless of depth. It then uses importlib.import_module to load each discovered package dynamically, building a registry of available checks without hard-coded import statements.

What naming convention must service modules follow?

Each module must reside at holehe/modules/<category>/<service>.py and define a function whose name exactly matches the filename. For instance, instagram.py must contain def instagram(email, client, out):. The get_functions utility extracts callables by splitting the dotted module name and indexing into module.__dict__ using the final segment.

Why does Holehe use Trio for executing modules?

Trio provides structured concurrency through nurseries, allowing Holehe to launch hundreds of service checks simultaneously while maintaining clean error propagation and cancellation semantics. The nursery.start_soon(launch_module, ...) pattern in maincore fires each check as an independent task that shares the httpx.AsyncClient for efficient connection pooling.

Where is the dynamic module loading logic implemented?

All runtime import and registration logic resides in holehe/core.py. Specifically, lines 37–47 handle recursive module discovery, lines 50–63 filter and extract callables, and lines 20–30 orchestrate concurrent execution. Individual service implementations live under holehe/modules/ (e.g., holehe/modules/social_media/instagram.py).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}