# How Holehe Discovers and Dynamically Loads Its Modules: Automated Plugin Architecture Explained

> Learn how Holehe dynamically discovers and loads its modules using Python's pkgutil and importlib for a plug-and-play architecture. Automated service checking explained.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-09-10

---

**Holehe leverages Python's `pkgutil.walk_packages` and `importlib` to recursively scan, import, and execute async service-checking modules at runtime, enabling a plug-and-play architecture where new services require zero configuration changes.**

The `megadose/holehe` repository implements a sophisticated extensibility system that automatically discovers and loads email-checking capabilities across dozens of web services. By utilizing Python's built-in import machinery rather than hardcoded registries, Holehe allows contributors to add new platforms simply by placing a properly structured Python file into the `holehe/modules/` directory tree.

## The Three-Phase Discovery Pipeline

Holehe's dynamic loading mechanism operates through three coordinated phases implemented in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py), transforming the static file system into a living registry of asynchronous checkers.

### Phase 1: Recursive Package Traversal with `import_submodules`

The discovery process begins in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) at lines 37-47 with the `import_submodules` function. This routine uses `pkgutil.walk_packages` to iterate over the `holehe.modules` package path, yielding every sub-package and module discovered.

For each item found, the function constructs the full dotted name (e.g., `holehe.modules.social_media.twitter`) and imports it using `importlib.import_module`. When the walker encounters a sub-package (indicated by `is_pkg=True`), the function recurses into that namespace to capture nested modules. This recursive walk ensures that categories like `social_media/` and `software/` are fully explored without manual path enumeration.

### Phase 2: Callable Extraction via `get_functions`

Once all modules reside in memory, `get_functions` (defined at lines 50-63 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)) filters the raw import dictionary to identify executable service checkers. The function applies a strict structural filter: only module paths containing more than three dotted components qualify (e.g., `holehe.modules.<category>.<module>`).

For each qualifying leaf module, the logic extracts the final component of the dotted path—which corresponds to the filename without extension—and retrieves that named attribute from the module's namespace. This convention enforces that the async function inside [`twitter.py`](https://github.com/megadose/holehe/blob/main/twitter.py) must be named `twitter`, creating a predictable mapping that `get_functions` uses to populate the `websites` list with callable objects.

### Phase 3: Asynchronous Execution in `maincore`

The orchestration layer resides in `maincore` at lines 105-108 of [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py). Here, the populated `websites` list enters a Trio nursery, where `launch_module` concurrently executes each discovered async function. The launcher awaits the module's main function—passing the target email, HTTP client, and output collector—and traps any exceptions, transforming them into uniform result dictionaries that indicate rate limits, errors, or account existence.

## Anatomy of a Service Module

To participate in the automatic discovery system, a module must adhere to a strict but simple contract. Consider [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py) (lines 5-8):

```python
async def twitter(email, client, out):
    # Implementation checks if email exists on Twitter

    # Appends result dict to 'out' list

    pass

```

The file must contain an async function matching its basename (`twitter`), accepting three parameters: the target email string, an HTTP client object, and a list to collect results. When `get_functions` processes this file, it extracts `twitter` from the module's `__dict__` and adds it to the execution queue. Supporting infrastructure like [`holehe/localuseragent.py`](https://github.com/megadose/holehe/blob/main/holehe/localuseragent.py) (custom user-agent strings) and [`holehe/instruments.py`](https://github.com/megadose/holehe/blob/main/holehe/instruments.py) (Trio progress meters) remain available to these dynamically loaded modules.

## Implementation Details and Code Examples

The following implementation mirrors the actual logic found in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py), demonstrating how to replicate Holehe's discovery pattern:

```python
import importlib
import pkgutil

def import_submodules(package):
    """Recursively import every submodule in the given package."""
    if isinstance(package, str):
        package = importlib.import_module(package)
    
    modules = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = f"{package.__name__}.{name}"
        modules[full_name] = importlib.import_module(full_name)
        if is_pkg:  # Recurse into sub-packages

            modules.update(import_submodules(full_name))
    return modules

```

Extracting the callable functions requires filtering for leaf modules and matching function names to filenames:

```python
def get_functions(modules):
    """Extract service-checking functions from imported modules."""
    functions = []
    for dotted_name, mod in modules.items():
        # Only keep leaf modules like holehe.modules.social_media.twitter

        if len(dotted_name.split(".")) > 3:
            fn_name = dotted_name.split(".")[-1]  # e.g., "twitter"

            if hasattr(mod, fn_name):
                functions.append(getattr(mod, fn_name))
    return functions

```

Execution within an async context uses Trio for concurrency:

```python
async def launch_module(module_func, email, client, out):
    try:
        await module_func(email, client, out)
    except Exception:
        # Uniform error handling

        out.append({
            "name": module_func.__name__,
            "domain": "...",  # Mapped elsewhere

            "rateLimit": False,
            "error": True,
            "exists": False,
        })

```

## Summary

- **Recursive discovery**: `import_submodules` in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) uses `pkgutil.walk_packages` to map the entire `holehe.modules` package tree without hardcoded paths.
- **Structural filtering**: `get_functions` enforces the `holehe.modules.<category>.<module>` convention by checking for more than three dotted name components.
- **Naming contract**: Each service module must define an async function matching its filename (e.g., `twitter` in [`twitter.py`](https://github.com/megadose/holehe/blob/main/twitter.py)) to be recognized.
- **Zero-configuration extensibility**: Adding new services requires only creating a file in `holehe/modules/<category>/`; the discovery system handles registration automatically.
- **Concurrent execution**: The `maincore` function passes discovered callables to a Trio nursery for parallel checking with standardized error handling.

## Frequently Asked Questions

### How does Holehe find new modules without explicit configuration?

Holehe treats the file system as the source of truth. The `import_submodules` function programmatically walks the `holehe/modules/` directory using Python's `pkgutil` module, importing every Python file it encounters. Because this happens at runtime, any new file added to the tree is automatically imported and made available without edits to a master registry or configuration file.

### What naming convention must service modules follow to be discovered?

Each module must reside at least three levels deep in the package hierarchy (`holehe.modules.category.module_name`) and contain an async function whose name exactly matches the filename. For example, [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py) must contain `async def twitter(...)`. The `get_functions` logic specifically looks for this correlation using the final component of the dotted module path.

### Why does Holehe require modules to have more than three components in their dotted name?

The check `len(dotted_name.split(".")) > 3` serves as a filter to distinguish leaf service modules from category packages and the root `holehe.modules` namespace itself. This ensures that only actual service implementations (like `holehe.modules.social_media.twitter`) are collected, while excluding intermediate packages (like `holehe.modules.social_media`) that contain no executable checker functions.

### How are errors handled when a dynamically loaded module fails during execution?

The `launch_module` function wraps each module call in a try-except block. If a module raises an exception, the error is caught and transformed into a standardized dictionary containing the function name, domain, and boolean flags indicating `error: True` and `exists: False`. This uniform error shape allows the main execution loop to continue processing other services even if one module crashes or encounters a network timeout.