How Holehe Tracks Scan Progress Using Trio Instruments

Holehe tracks scan progress by implementing a custom TrioProgress instrument that hooks into task lifecycle events, automatically incrementing a tqdm progress bar every time a website verification coroutine completes.

The open-source OSINT tool Holehe (megadose/holehe) performs asynchronous email verification across hundreds of websites simultaneously. To provide real-time visibility into batch operations, the tool implements a custom instrumentation layer that observes task completion without blocking the async event loop. This article examines the source code implementation in holehe/instruments.py and holehe/core.py to explain exactly how Holehe tracks scan progress.

The TrioProgress Instrument

Holehe leverages Trio's instrument API to monitor the lifecycle of asynchronous tasks. Rather than manually updating counters within each coroutine, the tool defines a custom TrioProgress class in holehe/instruments.py that inherits from trio.abc.Instrument:

class TrioProgress(trio.abc.Instrument):
    def __init__(self, total):
        self.tqdm = tqdm(total=total)

    def task_exited(self, task):
        if task.name.split(".")[-1] == "launch_module":
            self.tqdm.update(1)

The __init__ method initializes a tqdm progress bar with the total number of websites to scan. The critical task_exited method acts as a callback that Trio invokes whenever any task terminates. By checking if the task name ends with "launch_module", the instrument filters for specific website-check tasks and advances the bar by exactly one unit per completion.

Integrating the Instrument in holehe/core.py

The main scan orchestrator in holehe/core.py manages the instrument's lifecycle around the async nursery execution. Before launching tasks, the code instantiates TrioProgress with the website count and registers it with Trio's low-level instrumentation system:

instrument = TrioProgress(len(websites))
trio.lowlevel.add_instrument(instrument)

async with trio.open_nursery() as nursery:
    for website in websites:
        nursery.start_soon(launch_module, website, email, client, out)

trio.lowlevel.remove_instrument(instrument)

This pattern ensures that progress tracking starts before any work begins and cleans up immediately after all tasks finish. The add_instrument call activates the monitoring, while remove_instrument prevents memory leaks and detaches the tqdm handler once the scan completes.

Why the task_exited Hook Works

The task_exited callback fires regardless of whether a task succeeds or raises an exception. This guarantees that the progress bar advances even when a website check fails, maintaining an accurate count of processed sites versus remaining sites. The string filtering logic—task.name.split(".")[-1] == "launch_module"—precisely targets the coroutines spawned by nursery.start_soon(launch_module, ...) while ignoring internal Trio tasks or other background operations.

Summary

• Custom Instrumentation: Holehe uses a TrioProgress class inheriting from trio.abc.Instrument to observe task lifecycle events without modifying business logic.
• Automatic Updates: The task_exited hook increments a tqdm bar each time a launch_module task finishes, providing real-time feedback.
• Clean Lifecycle: The instrument is registered via trio.lowlevel.add_instrument before the scan and removed via remove_instrument after completion.
• Failure Resilience: Progress updates occur on task exit, ensuring the bar advances even when individual website checks raise exceptions.

Frequently Asked Questions

What is a Trio instrument?

A Trio instrument is a class inheriting from trio.abc.Instrument that registers callbacks for low-level task lifecycle events like task_spawned, task_exited, and before_task_step. Instruments observe the event loop without altering task execution, making them ideal for metrics collection, logging, and progress tracking in asynchronous applications.

How does Holehe know when a specific website check finishes?

Holehe checks the terminating task's name inside the task_exited method. When task.name.split(".")[-1] equals "launch_module", the instrument knows that a website verification coroutine has ended, distinguishing it from other system tasks. This naming convention matches the launch_module function passed to nursery.start_soon in holehe/core.py.

Can I modify the progress bar appearance when using TrioProgress?

Yes. The TrioProgress.__init__ method accepts standard tqdm arguments. You can customize the bar format, description, or output stream by modifying the initialization line in holehe/instruments.py or passing additional parameters when instantiating the class, though the current implementation uses tqdm's default configuration.

What happens if the scan is interrupted mid-execution?

If the scan receives a KeyboardInterrupt or encounters an unhandled exception, the async with trio.open_nursery() block exits, immediately triggering the trio.lowlevel.remove_instrument(instrument) cleanup line. While the tqdm bar may not reach 100%, the instrument is properly detached to prevent display corruption in subsequent operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}