# How Holehe Dynamically Discovers and Loads Modules: A Deep Dive into Its Extensible Architecture

> Discover how Holehe dynamically discovers and loads modules at runtime using Python's importlib and pkgutil. Learn about its extensible architecture and automatic service detection.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-09-01

---

**Holehe uses Python's `pkgutil.walk_packages` and `importlib.import_module` to dynamically discover and load service modules at runtime, eliminating hard-coded imports and enabling automatic detection of new services.**

Holehe's modular design allows contributors to add new service checks simply by dropping Python files into the `holehe/modules/` directory. This article explains exactly how the `megadose/holehe` repository implements dynamic module discovery, walking through the source code in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) that makes this extensibility possible.

## Understanding the Dynamic Module Discovery Pipeline

Holehe's discovery mechanism operates in three coordinated phases, all implemented in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py). The system uses standard Python import utilities rather than custom directory scanning, ensuring cross-platform compatibility and proper package resolution.

### Phase 1: Package Walk-through with `import_submodules`

The foundation of Holehe's discovery is the **`import_submodules`** function (lines 37-47 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)). This function leverages **`pkgutil.walk_packages`** to recursively traverse the `holehe.modules` package tree.

```python
def import_submodules(package, built_in=True):
    """Import all submodules of a module, recursively."""
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = package.__name__ + '.' + name
        results[full_name] = importlib.import_module(full_name)
    return results

```

For each discovered submodule, `pkgutil.walk_packages` yields:
- `loader`: The loader object for the module
- `name`: The short name of the submodule or subpackage
- `is_pkg`: Boolean indicating whether this is a package with submodules

The function constructs the **full dotted name** (e.g., `holehe.modules.social_media.twitter`) and imports it via **`importlib.import_module`**. This recursive approach ensures that even deeply nested service categories are automatically discovered.

### Phase 2: Function Extraction with `get_functions`

Once modules are loaded, the **`get_functions`** function (lines 50-63 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)) extracts the actual callable implementations. This function filters out internal packages and retrieves the async function that matches each service's name.

```python
def get_functions(modules, args):
    """Extract service functions from imported modules."""
    websites = []
    for module_name in modules:
        parts = module_name.split('.')
        # Skip internal packages (fewer than 4 components)

        if len(parts) < 4:
            continue
        # Extract the service name from the module path

        service_name = parts[-1]
        module_obj = modules[module_name]
        # Get the async function named after the service

        if hasattr(module_obj, service_name):
            websites.append(getattr(module_obj, service_name))
    return websites

```

The filtering logic (`len(parts) < 4`) excludes package infrastructure like `holehe.modules` or `holehe.modules.social_media` itself, keeping only leaf modules that contain actual service implementations.

### Phase 3: Asynchronous Execution with Trio

The extracted functions are executed concurrently using **Trio's nursery pattern** (lines 66-73 and 79-84 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)). The `maincore` function orchestrates this execution:

```python
async def maincore(email, args):
    modules = import_submodules("holehe.modules")
    websites = get_functions(modules, args)
    
    client = httpx.AsyncClient()
    out = []
    
    async with trio.open_nursery() as nursery:
        for site_fn in websites:
            nursery.start_soon(launch_module, site_fn, email, client, out)
    
    return out

async def launch_module(site_fn, email, client, out):
    """Wrapper to launch a module and capture its result."""
    await site_fn(email, client, out)

```

This design ensures that **hundreds of service checks run concurrently** without blocking, with each module receiving a shared HTTP client and output collection for efficiency.

## The Module Structure That Enables Discovery

For dynamic discovery to work, service modules must follow a **strict naming convention**. Each service module is a Python file located under `holehe/modules/<category>/<service>.py` that exports:

1. An **async function** named identically to the module file
2. The function signature: `async def <service>(email, client, out)`

Example from [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py):

```python
import httpx

async def twitter(email, client, out):
    """Check if email is registered on Twitter."""
    # Implementation details...

    out.append({
        "name": "twitter",
        "domain": "twitter.com",
        "method": "register",
        # ... result data

    })

```

The module path `holehe.modules.social_media.twitter` maps directly to the callable `twitter`, which `get_functions` extracts automatically. No registration boilerplate or explicit imports are required.

## Complete Discovery Workflow Example

Here's how the three phases connect in practice when Holehe runs an email check:

```python

# Phase 1: Discover all modules

>>> modules = import_submodules("holehe.modules")
>>> list(modules.keys())[:3]
[
    'holehe.modules.social_media.twitter',
    'holehe.modules.social_media.instagram',
    'holehe.modules.software.adobe'
]

# Phase 2: Extract service functions

>>> websites = get_functions(modules, args)
>>> [fn.__name__ for fn in websites[:3]]
['twitter', 'instagram', 'adobe']

# Phase 3: Execute concurrently (simplified)

>>> async with trio.open_nursery() as nursery:
...     for fn in websites:
...         nursery.start_soon(fn, "target@example.com", client, results)

```

## Why This Architecture Matters

The dynamic discovery approach in `megadose/holehe` provides several advantages over static imports:

- **Zero-configuration extensibility**: Adding [`holehe/modules/ecommerce/shopify.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/ecommerce/shopify.py) with an async `shopify` function automatically includes it in scans
- **No merge conflicts**: Contributors modify only their new service file, never touching central registry code
- **Lazy loading resilience**: Import failures in one module don't crash the entire application
- **Clean dependency boundaries**: Service categories map naturally to filesystem directories

According to the [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) source, this design has remained stable across versions because it relies on Python's standard library (`pkgutil`, `importlib`) rather than third-party plugin frameworks.

## Summary

- **Dynamic discovery** in Holehe relies on `pkgutil.walk_packages` in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) to traverse the `holehe.modules` package tree
- **`import_submodules`** (lines 37-47) builds a dictionary mapping full module names to imported module objects
- **`get_functions`** (lines 50-63) filters internal packages and extracts async functions matching their module names
- **Trio concurrency** executes discovered services without hard-coded orchestration logic
- New services require only a properly named `.py` file under `holehe/modules/` with an identically named async function

## Frequently Asked Questions

### How does Holehe avoid importing non-service modules?

`get_functions` filters modules by counting dot-separated components in the module name. Paths with fewer than 4 parts (e.g., `holehe.modules.social_media`) are skipped as package infrastructure, keeping only leaf modules like `holehe.modules.social_media.twitter`.

### What happens if a new module has an import error?

`importlib.import_module` raises `ImportError` during the `import_submodules` phase. Holehe doesn't implement explicit error handling in the discovery code, so a broken module will crash discovery unless fixed—a trade-off for simplicity.

### Can I add services to nested subdirectories?

Yes. `pkgutil.walk_packages` recurses automatically, so [`holehe/modules/productivity/collaboration/notion.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/productivity/collaboration/notion.py) would be discovered as `holehe.modules.productivity.collaboration.notion` provided you include [`__init__.py`](https://github.com/megadose/holehe/blob/main/__init__.py) files in each parent directory.

### Why use Trio instead of asyncio for concurrent execution?

The `megadose/holehe` codebase standardizes on Trio for structured concurrency. The `launch_module` wrapper (lines 79-84) provides a uniform interface for spawning each service check as a cancelable task with proper exception propagation—critical when running dozens of network-bound checks simultaneously.