How Holehe Tracks Progress During Scans: Trio Instrumentation and tqdm Integration

Holehe tracks scan progress by registering a custom TrioProgress instrument that hooks into Trio's task lifecycle events, updating a tqdm progress bar each time a launch_module coroutine completes.

Holehe is an open-source email reconnaissance framework that asynchronously checks hundreds of websites for account existence. When analyzing how Holehe tracks progress during scans, the implementation reveals an elegant combination of Trio’s low-level instrumentation API and the tqdm library to provide real-time, non-blocking progress feedback in the terminal.

The TrioProgress Instrument Architecture

Implementing the Instrument Class

In holehe/instruments.py, the TrioProgress class inherits from trio.abc.Instrument to gain access to Trio's internal task scheduling hooks. This design allows the progress tracker to observe task state changes without modifying the individual website checking logic.

When instantiated, the instrument creates a tqdm progress bar initialized with the total number of websites to scan:

import trio
from tqdm import tqdm

class TrioProgress(trio.abc.Instrument):
    def __init__(self, total):
        self.tqdm = tqdm(total=total)

The instrument stores a reference to the tqdm instance, which automatically handles terminal rendering, ETA calculations, and graceful resizing during the scan.

Tracking Task Lifecycle Events

The core tracking mechanism relies on the task_exited callback, which Trio invokes whenever an async task completes. The instrument filters specifically for the launch_module coroutine—the function responsible for checking a single website—to ensure the progress bar advances only when a site check finishes:

    def task_exited(self, task):
        # Only advance when a module finishes its `launch_module` call

        if task.name.split(".")[-1] == "launch_module":
            self.tqdm.update(1)

This filtering is critical because Trio's nursery may spawn auxiliary tasks; by checking task.name, the instrument distinguishes between actual website checks and internal bookkeeping operations.

Integrating Progress Tracking into the Core Scan Loop

Registering the Instrument

In holehe/core.py, the orchestration logic creates the instrument after building the list of target websites and registers it with Trio's low-level instrumentation stack before spawning any scan tasks:

from holehe.instruments import TrioProgress

# `websites` is the list of modules to scan

instrument = TrioProgress(len(websites))
trio.lowlevel.add_instrument(instrument)

The add_instrument call inserts the TrioProgress instance into Trio's global instrument list, enabling the task_exited callbacks to fire as nursery tasks complete.

Executing the Scan

All website checks run concurrently within a Trio nursery. As each launch_module coroutine exits, the instrument automatically increments the progress bar:

async with trio.open_nursery() as nursery:
    for website in websites:
        # each coroutine will call `launch_module`

        nursery.start_soon(launch_module, website, email, client, out)

Cleanup on Completion

After the nursery exits—indicating all website checks have finished—the instrument is explicitly removed from Trio's instrumentation stack to prevent memory leaks and ensure the progress bar finalizes properly:

# After all tasks finish

trio.lowlevel.remove_instrument(instrument)

This cleanup step is essential for long-running processes or when Holehe is used as a library within larger applications.

Complete Implementation Example

Putting the components together, the progress tracking flow in Holehe follows this pattern:

import trio
from holehe.instruments import TrioProgress
from holehe.core import launch_module

async def run_scan(websites, email, client, out):
    # Initialize progress tracking

    instrument = TrioProgress(len(websites))
    trio.lowlevel.add_instrument(instrument)
    
    try:
        async with trio.open_nursery() as nursery:
            for website in websites:
                nursery.start_soon(launch_module, website, email, client, out)
    finally:
        # Ensure instrument cleanup

        trio.lowlevel.remove_instrument(instrument)

When executed, the terminal displays a live progress bar similar to:

 45%|██████████▍          | 123/272 [00:12<00:15, 10.42it/s]

Summary

Holehe implements progress tracking through three key mechanisms:

• TrioProgress class in holehe/instruments.py bridges Trio's instrumentation API with tqdm visualization
• task_exited filtering specifically monitors launch_module completions to avoid counting auxiliary tasks
• Explicit lifecycle management in holehe/core.py registers the instrument before the scan and removes it after completion to prevent resource leaks

This architecture provides real-time feedback without blocking the async event loop, allowing Holehe to maintain high concurrency while keeping users informed of scan advancement.

Frequently Asked Questions

What is a Trio instrument and why does Holehe use one?

A Trio instrument is a class that inherits from trio.abc.Instrument and implements callback methods like task_spawned, task_exited, or before_io_wait. Holehe uses this pattern because it provides a clean separation between the scanning logic and progress reporting, allowing the progress bar to update automatically based on task lifecycle events rather than requiring manual updates within each website checking function.

How does Holehe distinguish between different types of async tasks?

The TrioProgress instrument inspects the task.name attribute in its task_exited method, specifically checking if the final segment of the task name equals "launch_module". This string filtering ensures that only completed website checks increment the progress bar, while ignoring auxiliary tasks spawned by Trio's internal machinery or connection pooling.

Can the progress bar be customized or disabled in Holehe?

Since TrioProgress initializes the tqdm bar in its __init__ method, modifying the progress display would require editing holehe/instruments.py to pass additional arguments to the tqdm constructor, such as disable=True to suppress output or custom bar_format strings. The instrument does not currently expose these options through the command-line interface in the core module.

Does progress tracking impact the scan performance?

The instrumentation overhead is negligible because Trio's instrument callbacks are lightweight function calls, and tqdm updates are throttled to the terminal's refresh rate. The filtering logic in task_exited involves only a string split and comparison operation, ensuring that the progress tracking does not materially slow down the concurrent website checks according to the source implementation in holehe/core.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}