# How Holehe Modules Are Discovered and Loaded Dynamically: A Deep Dive into the Python Import System

> Discover how Holehe modules are dynamically discovered and loaded at runtime using Python's pkgutil and importlib. Understand the import system without manual registration.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-08-23

---

**Holehe uses Python's `pkgutil.walk_packages` and `importlib` to recursively discover, import, and execute service-checking modules at runtime without manual registration.**

Holehe's modular architecture makes it trivial to add new service checks: drop a Python file into the correct directory and the framework handles the rest. This article explains exactly how Holehe discovers and loads modules dynamically, based on the source code in megadose/holehe.

## The Three-Stage Discovery Pipeline

The dynamic loading system in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) operates through three coordinated stages: recursive package traversal, callable extraction, and asynchronous execution.

### Stage 1: Recursive Package Walking with `import_submodules`

The entry point is `import_submodules` at lines 37-47 of [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py). This function leverages `pkgutil.walk_packages` to traverse the entire `holehe.modules` package tree.

```python
import importlib
import pkgutil

def import_submodules(package, recursive=True):
    """Recursively import every submodule in the given package."""
    if isinstance(package, str):
        package = importlib.import_module(package)
    
    modules = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = f"{package.__name__}.{name}"
        modules[full_name] = importlib.import_module(full_name)
        
        if is_pkg and recursive:
            modules.update(import_submodules(full_name))
    
    return modules

```

**Key implementation details:**

- **String-to-module conversion**: Accepts either a module object or a dotted package name
- **Full dotted names**: Constructs complete paths like `holehe.modules.social_media.twitter`
- **Controlled recursion**: The `recursive` flag enables deep traversal into category subdirectories
- **Eager import**: Calls `importlib.import_module` immediately, not lazily

### Stage 2: Extracting Callable Functions with `get_functions`

Raw module objects aren't directly executable. The `get_functions` implementation at lines 50-63 filters and transforms the discovered modules into callable async functions.

```python
def get_functions(modules, args=None):
    """Extract service-checking functions from imported modules."""
    functions = []
    
    for dotted_name in modules:
        # Filter: only leaf modules (depth > 3)

        # e.g., holehe.modules.social_media.twitter qualifies

        # but holehe.modules.social_media does not

        if len(dotted_name.split(".")) > 3:
            mod = modules[dotted_name]
            fn_name = dotted_name.split(".")[-1]  # "twitter"

            functions.append(mod.__dict__[fn_name])
    
    return functions

```

**The filtering logic enforces naming conventions:**

- Modules must be nested at least three levels deep (`holehe.modules.<category>.<module>`)
- The async function inside must match the final component of the dotted name
- This convention is what enables automatic execution without configuration files

### Stage 3: Asynchronous Execution via Trio Nursery

The final stage occurs in `maincore` (lines 105-108 of [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)). The collected functions are passed to a Trio nursery for concurrent execution.

```python
async def launch_module(module_func, email, client, out):
    """Execute a single service-checking module."""
    try:
        await module_func(email, client, out)
    except Exception:
        # Uniform error handling ensures consistent output format

        out.append({
            "name": module_func.__name__,
            "domain": "...",          # resolved from module metadata

            "rateLimit": False,
            "error": True,
            "exists": False,
            # ... additional result fields

        })

```

Each module receives three arguments:

| Parameter | Purpose |
|-----------|---------|
| `email` | The target email address to check |
| `client` | Pre-configured HTTP client (httpx) |
| `out` | List to append result dictionaries |

## Required Module Structure for Auto-Discovery

For a module to be discovered and executed, it must follow strict conventions. The [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py) file demonstrates the pattern:

```python

# File: holehe/modules/social_media/twitter.py

async def twitter(email, client, out):
    """
    Check if email is registered on Twitter.
    Must be async and match the filename.
    """
    # Perform HTTP request via provided client

    # Append result dictionary to out list

    out.append({
        "name": "twitter",
        "domain": "twitter.com",
        "method": "register",
        "frequent_rate_limit": False,
        "rateLimit": False,
        "exists": False,
        "emailrecovery": None,
        "phoneNumber": None,
        "others": None
    })

```

**Critical requirements:**

1. **File location**: `holehe/modules/<category>/<service_name>.py`
2. **Function name**: Must exactly match the filename (without `.py`)
3. **Function signature**: `async def <name>(email, client, out)`
4. **Side effects only**: Results appended to `out`; no return value used
5. **Async mandatory**: Function must be awaitable

## Package Structure Supporting Dynamic Loading

The discovery mechanism relies on specific package layout conventions:

```

holehe/
├── core.py              # Discovery and orchestration logic

├── modules/
│   ├── __init__.py      # Package marker (can be empty)

│   ├── social_media/    # Category subdirectory

│   │   ├── __init__.py
│   │   ├── twitter.py   # Service module (auto-discovered)

│   │   └── github.py
│   └── software/        # Another category

│       ├── __init__.py
│       └── docker.py
├── localuseragent.py    # HTTP client configuration

└── instruments.py       # Progress instrumentation for Trio

```

The [`__init__.py`](https://github.com/megadose/holehe/blob/main/__init__.py) files are essential—they mark directories as Python packages, enabling `pkgutil.walk_packages` to traverse them.

## Performance and Extensibility Characteristics

**Startup overhead**: Module discovery occurs once at import time, not per execution. The `import_submodules` call populates a module cache that `get_functions` consumes.

**Concurrency model**: All discovered modules execute concurrently through Trio's nursery pattern. No module blocks another regardless of network latency.

**Zero-registration extension**: Adding a service requires zero code changes to [`core.py`](https://github.com/megadose/holehe/blob/main/core.py). The filesystem is the sole source of truth.

**Memory footprint**: All modules are imported eagerly. For installations with many unused modules, this trades memory for simplicity.

## Summary

- **`import_submodules`** in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) uses `pkgutil.walk_packages` for recursive package traversal
- **`get_functions`** filters modules by depth and extracts filename-matching async functions
- **`launch_module`** executes each function through a Trio nursery with standardized error handling
- **Module conventions**: Files must live at `holehe/modules/<category>/<name>.py` with matching async function signatures
- **No registration required**: The filesystem structure alone determines what gets executed

## Frequently Asked Questions

### How does Holehe find new modules without configuration files?

Holehe treats the `holehe/modules/` directory as the configuration. The `import_submodules` function walks this package tree at runtime using `pkgutil.walk_packages`, importing every Python file it encounters. No central registry or manifest file exists—discovery is purely filesystem-based.

### Why must the async function name match the filename?

The `get_functions` extractor at line 58 of [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) derives the function name directly from the final component of the dotted module path. This convention eliminates ambiguity when multiple functions exist in a module and provides a predictable, zero-configuration execution target.

### Can modules be loaded lazily instead of at startup?

The current implementation in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) uses eager import via `importlib.import_module` during the discovery walk. While Python's import system supports lazy loading patterns, Holehe's design prioritizes fast failure detection and simple error handling over deferred loading.

### What happens if a module raises an exception during execution?

The `launch_module` function wraps each module call in a try-except block. Exceptions are caught and transformed into uniform result dictionaries with `"error": True`, ensuring one failing service doesn't crash the entire check run and preserving consistent output format across all modules.