# How Holehe's Async Module Execution with Trio Works: A Deep Dive into Parallel OSINT Scanning

> Explore Holehe's Trio async module execution for parallel OSINT scanning. Discover how it dynamically loads modules, shares HTTP clients, and tracks progress for efficient email-based checks.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: internals
- Published: 2026-08-20

---

**Holehe uses the Trio async framework to run hundreds of email-based checks in parallel, dynamically loading modules, sharing an HTTP client, and tracking progress with a custom instrument.**

When you supply an email address to Holehe, it verifies whether that address exists across a vast catalog of web services. To complete this scan quickly without blocking, the tool executes every service-specific check as an independent coroutine under Trio's supervision. The implementation in `megadose/holehe` demonstrates a clean pattern for high-throughput OSINT tooling built on structured concurrency.

## Dynamic Module Discovery and Function Extraction

Holehe does not hardcode its list of services. Instead, it discovers check modules at runtime.

In [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py), the `import_submodules()` function recursively imports every Python file under `holehe/modules/`:

```python

# From holehe/core.py, lines 37-47

def import_submodules(package, recursive=True):
    """Import all submodules of a module, recursively."""
    import importlib
    import pkgutil
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = package.__name__ + '.' + name
        results[full_name] = importlib.import_module(full_name)
        if recursive and is_pkg:
            results.update(import_submodules(full_name))
    return results

```

After importing, `get_functions()` (lines 50-63) walks these modules to collect callables that implement the checks. Each discovered function must match the signature `module(email, client, out)` and perform HTTP requests to determine account existence.

## Shared Async HTTP Client for Connection Reuse

Rather than creating a new HTTP session per check, Holehe instantiates a single `httpx.AsyncClient` that all coroutines share:

```python

# From holehe/core.py, lines 12-14

import httpx
client = httpx.AsyncClient(timeout=10)

```

This pattern maximizes connection reuse through HTTP keep-alive, reducing latency and avoiding connection pool exhaustion when hundreds of requests fire simultaneously.

## Trio Nursery: Structured Concurrency for Parallel Execution

The core parallel mechanism lives in `maincore()`. Holehe opens a Trio **nursery** — a context manager that supervises all spawned tasks and ensures none outlive the scope:

```python

# From holehe/core.py, lines 18-21

async with trio.open_nursery() as n:
    for website in websites:
        n.start_soon(launch_module, website, email, client, out)

```

Each call to `nursery.start_soon()` schedules `launch_module()` as a background task. Trio automatically interleaves these coroutines, yielding control whenever an `await` occurs (typically during HTTP I/O).

## Graceful Error Handling in launch_module

The `launch_module` wrapper (lines 66-78) isolates failures so one broken service cannot crash the entire scan:

```python

# Conceptual reconstruction from holehe/core.py

async def launch_module(module, email, client, out):
    try:
        await module(email, client, out)
    except Exception as e:
        # Record structured error without aborting

        out.append({
            "name": module.__name__,
            "domain": getattr(module, 'domain', 'unknown'),
            "exists": False,
            "emailrecovery": None,
            "phoneNumber": None,
            "others": None,
            "error": str(e)
        })

```

This defensive approach ensures partial results remain valid even when individual services timeout, block, or return unexpected data.

## Live Progress Tracking with a Custom Trio Instrument

Holehe implements `TrioProgress` in [`holehe/instruments.py`](https://github.com/megadose/holehe/blob/main/holehe/instruments.py) to drive a `tqdm` progress bar. Trio **instruments** are callbacks that hook into the runtime's task lifecycle events:

```python

# From holehe/instruments.py, lines 4-10

class TrioProgress:
    def __init__(self, total):
        self.pbar = tqdm(total=total, dynamic_ncols=True)

    def task_exited(self, task):
        # Called automatically by Trio when any task finishes

        self.pbar.update(1)

```

The instrument registers globally via `trio.lowlevel.add_instrument()` before the nursery opens and unregisters afterward. Each completed check updates the bar without explicit bookkeeping in the business logic.

## Result Aggregation and Cleanup

After the nursery exits (guaranteeing all tasks finished), Holehe performs orderly shutdown:

```python

# From holehe/core.py, lines 22-30

trio.lowlevel.remove_instrument(instrument)
await client.aclose()
out.sort(key=lambda r: r["name"])

# print or export sorted results...

```

The `out` list, populated concurrently by all modules, requires no explicit locking because Python's GIL protects list operations, and the design avoids shared-state mutations beyond append-only writes.

## Minimal Reproduction: Running Holehe Programmatically

```python
import trio
import httpx
from holehe.core import import_submodules, get_functions, launch_module
from holehe.instruments import TrioProgress

async def run_scan(email: str):
    """Mirror of holehe's internal execution flow."""
    # 1. Discover and load all service modules

    modules = import_submodules("holehe.modules")
    sites = get_functions(modules)  # list of coroutine functions

    
    # 2. Shared HTTP client

    client = httpx.AsyncClient(timeout=10)
    
    # 3. Result container and progress instrument

    out = []
    instrument = TrioProgress(len(sites))
    trio.lowlevel.add_instrument(instrument)
    
    # 4. Structured concurrency: spawn all checks

    async with trio.open_nursery() as n:
        for site in sites:
            n.start_soon(launch_module, site, email, client, out)
    
    # 5. Cleanup and return

    trio.lowlevel.remove_instrument(instrument)
    await client.aclose()
    return sorted(out, key=lambda r: r["name"])

# Execute

if __name__ == "__main__":
    results = trio.run(run_scan, "target@example.com")
    print(f"Found {sum(r['exists'] for r in results)} services")

```

## Summary

- **Dynamic loading**: `import_submodules()` and `get_functions()` discover service checks without hardcoding.
- **Structured concurrency**: Trio's nursery ensures all tasks complete or fail together with no orphaned coroutines.
- **Resource efficiency**: One `httpx.AsyncClient` serves all checks, pooling connections automatically.
- **Observable progress**: `TrioProgress` hooks task completion events to drive a live progress bar.
- **Fault isolation**: `launch_module` catches exceptions per-task, guaranteeing scan completion despite individual failures.

## Frequently Asked Questions

### What makes Trio better than asyncio for Holehe's use case?

Trio enforces **structured concurrency** through nurseries, making it impossible to lose track of spawned tasks. According to the holehe source code, this eliminates a class of bugs where background checks might continue past the main function's exit. The instrument API also provides cleaner observability hooks than asyncio's lower-level protocols.

### Can I adjust the number of concurrent requests?

Holehe does not implement explicit semaphore-based limiting. Trio schedules all tasks simultaneously, but the shared `httpx.AsyncClient` has internal connection pool limits. For stricter control, you would modify `maincore()` to use `trio.Semaphore` around the `start_soon()` calls.

### How does Holehe handle rate limiting by target services?

Individual modules handle rate limiting independently. The error-catching wrapper in `launch_module` ensures that HTTP 429 responses or timeouts from one service do not propagate. Some modules implement retry logic; others fail fast and record the error state in the output list.

### Is the `out` list thread-safe for concurrent appends?

Yes for CPython's current implementation. The Global Interpreter Lock (GIL) serializes `list.append()` operations. Trio runs single-threaded, so no additional synchronization is required. If Holehe migrated to multi-threaded execution, the append-only pattern would still remain safe, though explicit locking or `queue.Queue` would be recommended.