TrioProgress Instrument in holehe: How It Displays Progress for Async Site Checks

TL;DR: The TrioProgress instrument bridges trio's low-level instrumentation API with tqdm to display a live progress bar that updates each time a site check completes.

holehe performs bulk email/username reconnaissance across dozens of websites concurrently using trio for structured concurrency. To keep users informed without cluttering output, the project implements a lightweight custom instrument that visualizes completion progress. This article explains how TrioProgress works, where it's defined, and how it integrates with the async execution flow.

What Is the TrioProgress Instrument?

The TrioProgress instrument is a minimal class defined in holehe/instruments.py that implements trio's instrumentation API. Its sole purpose is to translate low-level task lifecycle events into visual progress updates via tqdm.

Unlike traditional progress tracking that requires manual updates inside each coroutine, this approach is non-intrusive—modules remain pure, and the instrument observes execution from outside.

How TrioProgress Displays Progress

The display mechanism follows a four-step flow:

1. Instantiation – Created with the total number of sites to check
2. Registration – Added to trio's instrument registry
3. Event handling – Reacts to task_exited events from the launch_module task
4. Cleanup – Removed after all checks complete

Core Implementation in instruments.py

The instrument class wraps a tqdm bar and filters events by task name:

# holehe/instruments.py

from tqdm import tqdm

class TrioProgress:
    def __init__(self, total):
        self.tqdm = tqdm(total=total, desc="Checking sites")

    def task_exited(self, task):
        # Only count completions from module launchers, not internal trio tasks

        if task.name.split(".")[-1] == "launch_module":
            self.tqdm.update(1)

The task_exited method is called by trio's runtime whenever any task finishes. The name filter ensures only actual site checks—not nursery management or other internal tasks—increment the bar.

Integration in core.py

The orchestration in holehe/core.py wires everything together:

# holehe/core.py (simplified)

import trio
from holehe.instruments import TrioProgress

async def run_checks(websites, email):
    # 1. Create instrument with total workload

    instrument = TrioProgress(len(websites))
    
    # 2. Register with trio's low-level API

    trio.lowlevel.add_instrument(instrument)

    async with trio.open_nursery() as nursery:
        for site in websites:
            nursery.start_soon(launch_module, site, email)

    # 4. Cleanup after all tasks complete

    trio.lowlevel.remove_instrument(instrument)

Key detail: The instrument is registered before opening the nursery and removed after it closes, ensuring full coverage of the execution window.

Practical Example

Here's a standalone demonstration mirroring holehe's pattern:

import trio
from tqdm import tqdm
from holehe.instruments import TrioProgress

async def dummy_check(site):
    """Simulate a site check."""
    await trio.sleep(0.3)
    return f"{site}: found"

async def main():
    sites = ["twitter.com", "github.com", "reddit.com", "gitlab.com", "instagram.com"]
    
    # Create and register progress instrument

    prog = TrioProgress(len(sites))
    trio.lowlevel.add_instrument(prog)

    results = []
    async with trio.open_nursery() as nursery:
        for site in sites:
            nursery.start_soon(dummy_check, site)

    trio.lowlevel.remove_instrument(prog)
    print(f"\nCompleted {len(sites)} checks")

trio.run(main)

Running this displays a tqdm bar that increments as each simulated check finishes—identical behavior to holehe's actual execution.

Why This Design Pattern Works

Advantage	Implementation Detail
Zero coupling	Modules don't import progress logic
Accurate counting	Filters by launch_module task name
Automatic cleanup	Trio guarantees task_exited fires
Minimal overhead	Single comparison per task exit

The instrument sits at the scheduling layer rather than the business logic layer, making it robust against changes to individual site check implementations.

Summary

• TrioProgress in holehe/instruments.py connects trio's instrumentation API to tqdm for live progress display
• It filters task_exited events by task name (launch_module) to count only relevant completions
• holehe/core.py instantiates it with the website count, registers it via trio.lowlevel.add_instrument(), and removes it after the nursery closes
• This pattern achieves progress tracking without modifying individual site check modules

Frequently Asked Questions

What is a trio instrument?

A trio instrument is a class implementing specific callback methods (task_spawned, task_exited, etc.) that trio's runtime invokes at task lifecycle events. TrioProgress uses this to observe completion without altering task code.

Why not call tqdm.update() inside each module?

Manually updating progress inside every launch_module would couple presentation logic to site checks. The instrument pattern keeps modules pure and allows centralized, consistent progress tracking.

Can TrioProgress track partial progress within a check?

No. TrioProgress only observes task-level events (start/exit), not internal progress. Individual site checks in holehe run quickly enough that per-check granularity isn't needed.

Does removing the instrument stop the progress bar?

The tqdm instance continues until garbage collected, but remove_instrument() stops further updates. The bar displays final completion state from the last update(1) call.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}