# How Holehe Dynamically Imports and Loads All Website Modules at Runtime: A Deep Dive into Its Plugin Architecture

> Explore how Holehe dynamically imports and loads website modules at runtime. Discover its zero-configuration plugin architecture and automatic module discovery.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-08-15

---

**Holehe discovers and loads every site-specific module automatically using a utility called `import_submodules` located in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py), enabling zero-configuration plugin discovery.**

The OSINT tool **holehe** by megadose supports hundreds of websites without requiring manual registration of each new module. This dynamic module loading system lets contributors add new site checks simply by creating Python files in the appropriate subdirectory. Understanding this mechanism reveals how to extend holehe or implement similar auto-discovery patterns in your own Python projects.

## The import_submodules Utility: Core of Dynamic Loading

All module discovery happens through a single function defined at **lines 37‑47 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)**. The `import_submodules` function implements a recursive package walker that transforms the static `holehe/modules` directory tree into a dictionary of loaded module objects.

### How the Discovery Works

The utility combines two standard library modules: `pkgutil` for package introspection and `importlib` for runtime importing:

```python
import importlib, pkgutil

def import_submodules(package, recursive=True):
    """Import every submodule of *package* and return a dict."""
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = f"{package.__name__}.{name}"
        results[full_name] = importlib.import_module(full_name)
        if recursive and is_pkg:
            results.update(import_submodules(full_name))
    return results

```

**`pkgutil.walk_packages`** yields three-tuples for every entry in the package: the loader object, the module name, and a boolean `is_pkg` flag. This iteration covers both `.py` files and subdirectories containing [`__init__.py`](https://github.com/megadose/holehe/blob/main/__init__.py).

### Building the Full Module Name

For each discovered entry, the function constructs the **fully qualified dotted name** by concatenating `package.__name__ + '.' + name`. This preserves the nested structure so that [`twitter.py`](https://github.com/megadose/holehe/blob/main/twitter.py) inside `holehe/modules/social_media/` becomes `holehe.modules.social_media.twitter`.

## Recursive Loading of Nested Packages

The `recursive=True` default ensures holehe handles arbitrary directory depth. When `is_pkg` is `True`, `import_submodules` calls itself on that sub-package and merges results with `results.update()`.

This design means the directory structure directly determines the module hierarchy:

```

holehe/modules/
├── social_media/
│   ├── __init__.py
│   ├── twitter.py          → holehe.modules.social_media.twitter
│   └── instagram.py        → holehe.modules.social_media.instagram
├── shopping/
│   ├── __init__.py
│   ├── amazon.py           → holehe.modules.shopping.amazon
│   └── ebay.py             → holehe.modules.shopping.ebay
└── __init__.py

```

No configuration file maps these paths to module names—the filesystem is the source of truth.

## Runtime Invocation in maincore

The dynamic import system activates during the main execution flow. At **lines 206‑208 in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)**, the `maincore` function orchestrates the loading:

```python
modules = import_submodules("holehe.modules")
websites = get_functions(modules, args)

```

**The two-phase process:**

1. **`import_submodules("holehe.modules")`** returns a dictionary where keys are full module names and values are actual module objects. Every `.py` file under `holehe/modules` is now imported and available.

2. **`get_functions(modules, args)`** filters and transforms these modules into callable coroutines. It scans each module for functions matching the expected signature and selects those relevant to the command-line arguments.

## From Module to Callable: Extracting Site Checkers

Each site module exports an **async function** with the same name as the module file. For example, [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py) contains:

```python
async def twitter(email, client, out):
    # HTTP request to Twitter's password recovery endpoint

    # ...

    out.append({
        "name": "twitter",
        "domain": "twitter.com",
        "exists": True,
        "emailrecovery": "t****@gmail.com",
        # ...

    })

```

The `get_functions` utility inspects the imported modules using introspection, identifies these async functions, and returns them as a list of callables. These are then launched concurrently with **Trio** for efficient asynchronous execution against the target email address.

## Performance and Scalability Characteristics

The `import_submodules` approach carries specific runtime implications:

- **Import-time overhead** occurs once at startup. All modules load regardless of whether they'll be used for a particular scan.
- **Memory footprint** includes every site module's code and dependencies in the Python process.
- **Zero runtime discovery cost** during actual OSINT checks—modules are already loaded and referenced.

For a tool like holehe with hundreds of modules, this tradeoff favors simplicity and contributor ergonomics over minimal memory usage.

## Extending Holehe: Adding New Site Modules

The dynamic import architecture enables **frictionless contribution**. To add support for a new website:

1. Create a Python file under an appropriate category in `holehe/modules/` (or create a new category directory with [`__init__.py`](https://github.com/megadose/holehe/blob/main/__init__.py)).

2. Implement an async function matching the expected signature:

```python

# holehe/modules/social_media/newsite.py

import httpx

async def newsite(email, client, out):
    """Check if email is registered on newsite.com."""
    try:
        r = await client.get(
            f"https://newsite.com/api/check_email?email={email}"
        )
        exists = r.status_code == 200
    except Exception:
        exists = False
    
    out.append({
        "name": "newsite",
        "domain": "newsite.com",
        "exists": exists,
        "emailrecovery": None,
        "phoneNumber": None,
        "others": None,
    })

```

3. No registration step required—the next run of holehe automatically includes `newsite` in its module dictionary.

## Summary

- **`import_submodules`** in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) (lines 37‑47) implements recursive dynamic importing using `pkgutil.walk_packages` and `importlib.import_module`.
- **Package tree walking** generates full dotted names for every Python file under `holehe/modules`, enabling unlimited nesting.
- **Runtime loading** happens in `maincore` (lines 206‑208), producing a dictionary of all site modules without manual registration.
- **`get_functions`** extracts async coroutines from loaded modules, preparing them for concurrent Trio execution.
- **Contributor workflow** requires only adding a Python file—no configuration changes or registry updates.

## Frequently Asked Questions

### How does holehe find modules without a central registry?

Holehe treats the filesystem as the registry. The `pkgutil.walk_packages` function enumerates all subdirectories and Python files under `holehe/modules`, and `importlib.import_module` loads each one by its computed dotted name. This eliminates the need for manual list maintenance.

### Can holehe load modules from outside its package directory?

Not with the default `import_submodules` implementation. The function specifically walks `package.__path__`, which is constrained to the `holehe/modules` directory tree. Extending to external plugin directories would require modifying the path or implementing a separate plugin discovery mechanism.

### What happens if a module has syntax errors or import failures?

`importlib.import_module` will raise an `ImportError` or `SyntaxError` during `import_submodules` execution, causing holehe to crash at startup. The current implementation does not include error isolation for individual module failures.

### Why use recursive loading instead of flat module imports?

Recursion preserves the categorical organization of site modules (social_media, shopping, etc.) while keeping the loading code simple. It also ensures that future category additions require zero changes to the import machinery—just create a new subdirectory.