# How Holehe Categorizes Target Websites into Modules: Dynamic Discovery Explained

> Learn how Holehe categorizes target websites into modules dynamically. Discover automatic module organization via pkgutil.walk_packages for efficient vulnerability scanning.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-09-01

---

**Holehe organizes target websites into functional modules using a filesystem-based category structure under `holehe.modules`, with automatic discovery via `pkgutil.walk_packages` and no manual registration required.**

The open-source OSINT tool [Holehe](https://github.com/megadose/holehe) (by [megadose](https://github.com/megadose)) checks whether an email address is registered on hundreds of websites. Its modular architecture lets developers add new services without touching core code. This article explains how Holehe categorizes target websites into modules based on the actual source implementation.

## Filesystem-Based Module Categorization

Holehe uses **Python package hierarchy** as its categorization system. The `holehe/modules/` directory contains sub-packages named after vertical categories, with each website implemented as a standalone Python file.

```

holehe/
 └─ modules/
     ├─ social_media/
     │   ├─ facebook.py
     │   ├─ twitter.py
     │   └─ ...
     ├─ music/
     │   ├─ spotify.py
     │   └─ ...
     ├─ transport/
     │   ├─ blablacar.py
     │   └─ ...
     ├─ shopping/
     ├─ crm/
     └─ ...

```

Each `.py` file exports **one asynchronous function** matching the filename. For example, [`facebook.py`](https://github.com/megadose/holehe/blob/main/facebook.py) contains `async def facebook(email, client, out)`.

## Dynamic Module Discovery in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)

The categorization system relies on two key functions in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) that discover modules at runtime without explicit imports.

### `import_submodules` Walks the Package Tree

```python

# From holehe/core.py lines 37-47

def import_submodules(package: str) -> tuple:
    """Import all submodules of a package, recursively."""
    package = importlib.import_module(package)
    return {
        name: importlib.import_module(f"{package.__name__}.{name}")
        for _, name, _ in pkgutil.walk_packages(package.__path__)
    }

```

This uses `pkgutil.walk_packages` to traverse every subdirectory under `holehe.modules` and eagerly imports each module found.

### `get_functions` Filters Valid Targets

```python

# From holehe/core.py lines 50-63

def get_functions(modules: dict) -> list:
    """Extract callable functions from imported modules."""
    functions = []
    for module_path, module in modules.items():
        parts = module_path.split('.')
        # Only paths like: holehe.modules.<category>.<site>

        if len(parts) > 3:
            for name, obj in inspect.getmembers(module):
                if callable(obj) and name == parts[-1]:
                    functions.append(obj)
    return functions

```

The **three-component path check** (`holehe.modules.<category>.<site>`) enforces the categorization convention. Any module deeper than `holehe.modules` is treated as a target website.

## How New Categories and Sites Are Added

The filesystem-based approach enables **zero-registration extensibility**:

1. Create a new folder under `holehe/modules/` for a new category
2. Add a Python file with an async function matching the filename
3. Restart Holehe — the discovery logic automatically includes it

This design removes the need for:
- Central registries or configuration files
- Decorator-based registration
- Manual import statements in core code

## Code Examples: Working with Categorized Modules

### Running a Single Category Module

```python
import trio
import httpx
from holehe.modules.social_media.facebook import facebook

async def check_facebook():
    email = "target@example.com"
    out = []
    async with httpx.AsyncClient() as client:
        await facebook(email, client, out)
    print(out)  # [{'name': 'facebook', 'exists': ..., ...}]

trio.run(check_facebook)

```

### Running All Discovered Modules Concurrently

```python
import trio
import httpx
from holehe.core import import_submodules, get_functions

async def run_full_probe(email: str):
    # Dynamically discover all categorized modules

    all_modules = import_submodules('holehe.modules')
    functions = get_functions(all_modules)
    
    out = []
    async with httpx.AsyncClient() as client:
        async with trio.open_nursery() as nursery:
            for func in functions:
                nursery.start_soon(func, email, client, out)
    
    return out

# Execute probe across all categories

results = trio.run(run_full_probe, "target@example.com")

```

## Key Source Files for Module Categorization

| File | Purpose |
|------|---------|
| [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) | Implements `import_submodules()` and `get_functions()` for dynamic discovery |
| [`holehe/modules/__init__.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/__init__.py) | Empty package marker; enables subpackage imports |
| [`holehe/modules/social_media/facebook.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/facebook.py) | Example module showing required async signature |
| `holehe/modules/` subdirectories | Category folders (`music/`, `transport/`, `shopping/`, etc.) |

## Summary

- **Holehe categorizes target websites** using Python package subdirectories as category folders
- **Dynamic discovery** via `pkgutil.walk_packages` in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) eliminates manual registration
- **Path depth validation** (`holehe.modules.<category>.<site>`) determines valid target modules
- **Zero-config extensibility** — adding websites requires only new files, no core changes
- **Uniform execution interface** — every module exports one async function with identical signature

## Frequently Asked Questions

### How does Holehe know which modules to load without a central registry?

Holehe scans the `holehe.modules` package tree at runtime using `pkgutil.walk_packages` in `import_submodules()`. Any Python file found in a subfolder is automatically imported and evaluated by `get_functions()` if its import path has more than three components.

### Can I add my own custom category folder?

Yes. Create any new directory under `holehe/modules/` (for example, `holehe/modules/gaming/`). Place Python files inside with async functions matching their filenames. The discovery logic will include them automatically on the next run.

### What happens if two modules have the same name in different categories?

The full dotted path distinguishes them. `holehe.modules.social_media.twitter` and `holehe.modules.news.twitter` would both be valid, though this naming collision is discouraged for clarity.

### Does Holehe support nested subcategories beyond one level?

No. The `get_functions()` check requires exactly `holehe.modules.<category>.<site>` (four path components). Nested structures like [`holehe/modules/social_media/video/tiktok.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/video/tiktok.py) would be ignored by the current discovery logic.