How Holehe Implements Its Async Progress Bar: A Deep Dive into Trio's Instrumentation API

Holehe implements its progress bar by wrapping tqdm in a custom trio.abc.Instrument subclass that updates on every completed site check without blocking asynchronous execution.

The open-source OSINT tool holehe by megadose scans hundreds of websites for account correlations. Its progress bar implementation is a textbook example of integrating synchronous UI libraries with structured concurrency frameworks. This article explains how the progress bar implementation in Holehe works, complete with source code references from the repository.

The Core Challenge: Progress Bars in Async Code

Standard progress bar libraries like tqdm assume synchronous iteration. Holehe uses Trio for structured concurrency, launching dozens of simultaneous network requests. The solution hinges on Trio's instrumentation API—a low-level hook system that observes task lifecycle events.

The TrioProgress Instrument Class

In holehe/instruments.py, Holehe defines a custom TrioProgress class inheriting from trio.abc.Instrument:

class TrioProgress(trio.abc.Instrument):
    def __init__(self, total):
        self.tqdm = tqdm(total=total)

The constructor receives the total number of sites to check and initializes a tqdm bar with that count.

Tracking Task Completion

The critical method is task_exited, which Trio calls whenever any task finishes:

def task_exited(self, task):
    if task.name.split(".")[-1] == "launch_module":
        self.tqdm.update(1)

This filters for tasks named launch_module—the specific coroutine that executes each site's check—ignoring internal Trio tasks. When a match occurs, self.tqdm.update(1) advances the visual bar by one step.

Registering the Instrument at Runtime

In holehe/core.py, the main execution flow instantiates and registers the instrument:

instrument = TrioProgress(len(websites))
trio.lowlevel.add_instrument(instrument)
...
trio.lowlevel.remove_instrument(instrument)

The registration happens before the nursery opens, and removal occurs after all scans complete. This ensures the progress bar receives every task exit event during the concurrent execution window.

Complete Execution Flow

1. Initialization: TrioProgress creates a tqdm bar sized to len(websites)
2. Registration: trio.lowlevel.add_instrument() subscribes the object to task events
3. Concurrent execution: trio.open_nursery() spawns launch_module tasks for each site
4. Progress updates: Each completing task triggers task_exited, updating the bar
5. Cleanup: trio.lowlevel.remove_instrument() unsubscribes the progress tracker

This design provides thread-safe progress tracking without modifying the scanning logic or introducing locks.

Practical Code Example

Here's how to replicate Holehe's pattern in your own Trio applications:

from holehe.instruments import TrioProgress
import trio

# Configure for your workload size

progress = TrioProgress(total=42)

# Register before starting concurrent work

trio.lowlevel.add_instrument(progress)

async def probe_site(name):
    # Your async operation here

    await trio.sleep(0.1)

async def run_all():
    async with trio.open_nursery() as nursery:
        for site in sites:
            nursery.start_soon(probe_site, site)

trio.run(run_all)

# Always clean up instruments

trio.lowlevel.remove_instrument(progress)

Key Files and Architecture

File	Purpose	Lines
holehe/instruments.py	Defines TrioProgress class with tqdm integration and task_exited callback	1-10
holehe/core.py	Orchestrates instrument lifecycle: instantiation, registration, execution, removal	16-22

Why This Approach Works

• Non-blocking: The instrument runs on Trio's internal scheduling, never pausing network I/O
• Precise tracking: Filtering by task name (launch_module) avoids counting internal Trio machinery
• Clean separation: Progress logic lives in instruments.py, keeping core.py focused on orchestration
• Zero dependencies beyond tqdm and Trio: No custom C extensions or platform-specific code

Summary

• Holehe's progress bar combines tqdm with Trio's Instrument API for async-safe updates
• The TrioProgress class in holehe/instruments.py implements task_exited to increment on each finished site check
• Registration in holehe/core.py uses trio.lowlevel.add_instrument() and remove_instrument() for precise lifecycle control
• Task name filtering ensures only meaningful work updates the progress bar, not internal framework tasks
• This pattern is reusable for any Trio application needing visual progress without blocking concurrency

Frequently Asked Questions

Why doesn't Holehe use asyncio instead of Trio?

Holehe uses Trio for its structured concurrency guarantees—nurseries ensure all spawned tasks complete before exiting, preventing the "fire and forget" bugs common in asyncio. The instrumentation API is also more mature in Trio, making the progress bar implementation cleaner.

Can this pattern work with asyncio's TaskGroup?

Not directly. Asyncio lacks an equivalent low-level instrumentation API. You would need to manually wrap coroutines or use asyncio.gather() with a callback mechanism, which is less elegant than Trio's automatic task_exited notifications.

Does the progress bar slow down the scanning?

Negligibly. The task_exited callback performs only a string split, comparison, and tqdm.update(1) call—all O(1) operations. The actual bar rendering happens in tqdm's internal thread, decoupled from Trio's event loop.

What happens if a task crashes?

Trio still calls task_exited for crashed tasks, so the progress bar advances correctly. Error handling occurs separately in the nursery's exception propagation, which doesn't interfere with instrument callbacks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}