Understanding the Architecture of the holehe OSINT Tool: Modular Design and Async Execution

The holehe OSINT tool employs a three‑pillar architecture built on dynamic module loading, structured concurrent execution via the Trio async framework, and standardized result aggregation that allows it to check hundreds of services without hard‑coded service lists.

holehe is a Python‑based OSINT (Open‑Source Intelligence) utility designed to verify whether an email address is registered across hundreds of online platforms. Unlike traditional scrapers that embed site logic directly in the main codebase, holehe implements a plug‑and‑play modular architecture where each service checker lives as an independent async function. The core engine in holehe/core.py automatically discovers these modules at runtime, executes them concurrently, and collates results into a unified report.

Core Architectural Pillars

Dynamic Module Loading

The framework eliminates manual registration through runtime introspection. The import_submodules() function in holehe/core.py (lines 37–47) recursively walks the holehe.modules package tree using pkgutil.walk_packages(). It imports every submodule—whether categorized under mails/, social/, or programming/—and returns a mapping of fully‑qualified names to module objects.

Subsequently, get_functions() filters this mapping to identify async callables that implement the standard site‑check signature. This design respects CLI flags such as --no-password-recovery, dynamically excluding specific categories without code changes.

Concurrent Execution with Trio

Performance relies on structured concurrency rather than threaded parallelism. The maincore() function initializes a shared httpx.AsyncClient for connection pooling and TLS reuse, then opens a Trio nursery via trio.open_nursery(). For every discovered module, the nursery schedules launch_module(), a coroutine defined in holehe/core.py (lines 66–78) that executes the site‑specific check.

Each module—exemplified by google() in holehe/modules/mails/google.py—receives three arguments: the target email, the shared HTTP client, and a thread‑safe result list. The launch_module() wrapper encloses each execution in a try/except block, ensuring isolated failures are recorded as errors rather than aborting the entire scan.

Progress Reporting and Result Aggregation

Real‑time feedback is implemented through Trio's instrumentation API. The TrioProgress class in holehe/instruments.py attaches to the nursery, updating a tqdm progress bar as tasks start and complete. Once all checks finish, print_result() in holehe/core.py (around lines 6–49) alphabetizes the results, applies ANSI color coding for terminal display, and optionally serializes the data to CSV via export_csv().

Execution Flow from Entry to Output

CLI Initialization and Validation

The entry point main() delegates immediately to trio.run(maincore), establishing an async runtime before any I/O occurs. maincore() parses arguments using argparse, validates email format with regex, and instantiates the HTTP client with configurable timeouts. This centralized initialization ensures all modules share the same session configuration.

Module Discovery and Filtering

During the setup phase, import_submodules("holehe.modules") discovers every Python file in the modules hierarchy. get_functions() then inspects these modules for async functions, building an execution list that respects user‑supplied inclusion or exclusion filters. This dynamic discovery means the tool requires no central registry; adding a new service only requires placing a file in the correct directory.

Isolated Task Execution

The nursery schedules launch_module(site, email, client, out) for each target service. This coroutine invokes the site‑specific logic and appends a standardized dictionary to the shared out list. Because Trio nurseries enforce structured concurrency, all tasks complete before the program exits, and exceptions in individual modules cannot leak to crash the main process.

Module Design and Implementation Patterns

Anatomy of a Site Checker

Every module follows a strict async signature: async def servicename(email, client, out). For example, holehe/modules/mails/google.py demonstrates the canonical pattern:

• Defines local variables for name and domain
• Rotates user‑agents imported from holehe/localuseragent.py to avoid detection
• Performs one or more await client.get() or await client.post() requests
• Parses HTTP responses to determine if the email is registered, rate‑limited, or invalid
• Appends a result dictionary to the out list

Standardized Result Schema

Modules populate a strict dictionary structure consumed by the core:

• name: Service identifier string
• domain: Target domain (e.g., google.com)
• rateLimit: Boolean indicating if the request was throttled
• exists: Boolean indicating account registration status
• emailrecovery, phoneNumber, others: Optional metadata fields for recovery information

This schema decouples the presentation layer from data collection, enabling consistent CSV exports and terminal formatting regardless of the service's underlying API structure.

Practical Usage and Extension

Command-Line Interface

Basic invocation requires only an email address:

# Basic scan

holehe user@example.com

# Filter to registered accounts only

holehe user@example.com --only-used

# Export to CSV

holehe user@example.com --csv

Programmatic Library Integration

Developers can bypass the CLI and embed holehe directly into security pipelines:

import trio
import httpx
from holehe.core import import_submodules, get_functions, launch_module

async def osint_check(target_email: str):
    """Execute holehe checks programmatically."""
    modules = import_submodules("holehe.modules")
    sites = get_functions(modules)
    
    async with httpx.AsyncClient(timeout=10) as client:
        results = []
        async with trio.open_nursery() as nursery:
            for site in sites:
                nursery.start_soon(launch_module, site, target_email, client, results)
    
    return sorted(results, key=lambda x: x["name"])

# Execute

if __name__ == "__main__":
    results = trio.run(osint_check, "test@example.com")
    for r in results:
        print(f"{r['name']}: {r['exists']}")

Adding Custom Modules

Extending the tool requires no modifications to holehe/core.py. Create a new file under holehe/modules/social/newservice.py implementing the standard signature:

async def newservice(email, client, out):
    name = "newservice"
    domain = "newservice.com"
    
    headers = {
        "User-Agent": "Mozilla/5.0...",
        "Accept": "application/json"
    }
    
    try:
        response = await client.get(
            f"https://{domain}/api/check?email={email}",
            headers=headers
        )
        exists = response.status_code == 200
    except Exception:
        exists = False
    
    out.append({
        "name": name,
        "domain": domain,
        "rateLimit": False,
        "exists": exists,
        "emailrecovery": None,
        "phoneNumber": None,
        "others": None
    })

The next execution automatically discovers and runs the new module.

Summary

• Dynamic Discovery: holehe/core.py uses import_submodules() to recursively load all Python files under holehe/modules/, eliminating manual service registration
• Structured Concurrency: The Trio nursery model manages hundreds of simultaneous HTTP requests through a shared httpx.AsyncClient, with launch_module() providing fault isolation
• Standardized Contracts: Every module implements the same async signature and returns a uniform dictionary, enabling consistent result processing and formatting
• Zero‑Config Extensibility: New OSINT targets require only a new file in the modules directory following the established async pattern
• Instrumentation: TrioProgress in holehe/instruments.py provides real‑time progress feedback without blocking the async event loop

Frequently Asked Questions

What async framework does holehe use for concurrent execution?

holehe uses the Trio library for structured concurrency. According to the source code in holehe/core.py, the main() entry point calls trio.run(maincore), and maincore() uses trio.open_nursery() to schedule launch_module() tasks. This approach provides deterministic cleanup and superior error handling compared to traditional asyncio for I/O‑bound OSINT workloads.

How does holehe discover new modules without code changes?

The import_submodules() function in holehe/core.py (lines 37–47) recursively walks the holehe.modules package using pkgutil.walk_packages(). It imports every submodule dynamically, and get_functions() extracts the async callables. Because discovery happens at runtime, placing any new Python file implementing the standard signature under holehe/modules/ automatically makes it available on the next scan.

Can holehe be used as a Python library instead of a CLI tool?

Yes. Rather than invoking the main() function, developers can import import_submodules, get_functions, and launch_module from holehe/core.py, then manually construct a Trio nursery and httpx.AsyncClient. This allows integration into larger security frameworks, custom filtering of results, or embedding within automated OSINT pipelines.

What prevents a single failing module from crashing the entire scan?

The launch_module() coroutine in holehe/core.py wraps each site check in a try/except block. If a module raises an exception—whether from network timeouts, HTML parsing errors, or API changes—the error is caught and recorded. The Trio nursery continues executing remaining tasks, ensuring comprehensive coverage even when individual services are unstable or offline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}