# Performance Considerations for Concurrently Checking 120+ Websites with Holehe

> Discover performance considerations for concurrently checking 120+ websites with Holehe. Learn to tune connection limits, timeouts, and module selection for high throughput email verification.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: performance
- Published: 2026-08-15

---

**Holehe achieves high-throughput email verification across hundreds of sites by using Trio's structured concurrency with a shared HTTPX async client, but optimal performance requires tuning connection limits, timeouts, and module selection.**

When scanning an email address against 120 or more websites, understanding Holehe's concurrency model is essential for maximizing speed without triggering rate limits. This guide examines the architectural decisions in `megadose/holehe` that affect performance and provides actionable optimizations based on the source code implementation.

## How Holehe Handles Concurrent Requests

Holehe's concurrency engine resides in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py). The tool orchestrates all site checks through **Trio nurseries** and a **shared asynchronous HTTP client**.

### The Async Client Architecture

In `core.py:L13-L14`, Holehe instantiates a single `httpx.AsyncClient`:

```python
client = httpx.AsyncClient(timeout=timeout)

```

This single client is passed to every module, enabling **connection reuse** across all 120+ site checks. The HTTPX default limits (approximately 100 concurrent connections) cap how many simultaneous TCP handshakes occur.

To increase throughput in a custom fork, override these defaults:

```python
import httpx

limits = httpx.Limits(
    max_keepalive_connections=200,
    max_connections=200
)
client = httpx.AsyncClient(timeout=10, limits=limits)

```

### Trio Nursery Task Dispatch

The parallel execution happens in `core.py:L18-L21`:

```python
async with trio.open_nursery() as n:
    for function in get_functions(modules, args):
        n.start_soon(launch_module, function, args.email, client, out)

```

**Every module starts immediately** as a separate coroutine. With 120+ websites, this creates 120+ concurrent tasks competing for the event loop's attention and the HTTP connection pool.

## Key Bottlenecks When Scaling to 120+ Sites

### Connection Pool Saturation

With HTTPX defaults, the 101st concurrent request queues until a connection frees. For 120 sites, this creates minor queuing; for 300 (Holehe's full module set), it becomes significant.

**Symptoms:** Progress bar moves in batches rather than steadily.

**Mitigation:** Filter modules to only necessary sites using `--no-password-recovery` (`-NP`) or a custom module list.

### Timeout Configuration

The `--timeout` parameter (`core.py:L94-L96`) bounds each request:

```bash

# Aggressive timeout for fast networks

holehe user@example.com -T 5

# Generous timeout for VPN or high-latency paths

holehe user@example.com -T 20

```

A single hanging site with a 10-second default timeout delays result collection for that slot. With 120 sites, multiple slow endpoints compound this effect.

### Rate Limiting and 429 Responses

Modules return `rateLimit=True` when detecting HTTP 429 or equivalent (`core.py:L22-L26`). Holehe continues other tasks, but the underlying TCP connection was already consumed.

**Detection:** Use `--no-clear` to preserve the progress bar and watch for stalls.

**Response:** Consider a custom wrapper with controlled delays:

```python
import trio

async def throttled_launch(function, email, client, out):
    await trio.sleep(0.1)  # 100ms stagger between starts

    await launch_module(function, email, client, out)

```

## CPU and Memory Characteristics

Holehe's workload is **I/O-bound, not CPU-bound**. Typical CPU usage stays below 5% on modern hardware because the event loop spends nearly all time awaiting network responses.

**Implications:**

- Adding CPU-intensive parsing (e.g., complex HTML extraction) disrupts this balance
- Memory usage scales with the number of concurrent responses held in memory
- The `TrioProgress` instrument (`instruments.py:L8-L10`) adds negligible overhead via `tqdm` updates

## Practical Optimization Strategies

### 1. Reduce Module Count

Holehe discovers ~300 modules by default via `import_submodules("holehe.modules")` (`core.py:L36-L47`). For 120-site targets, explicitly select relevant modules:

```python
from holehe.core import import_submodules, get_functions

modules = import_submodules("holehe.modules")

# Filter to specific sites

target_sites = ['github', 'twitter', 'instagram', 'linkedin']
filtered = {k: v for k, v in modules.items() if any(t in k for t in target_sites)}

```

### 2. Monitor with Instruments

The `TrioProgress` class provides real-time visibility. A flat progress bar indicates mass rate-limiting or network congestion.

```bash
holehe user@example.com --no-clear

```

### 3. Programmatic Control

For production scanning, replicate Holehe's core logic with custom constraints:

```python
import httpx
import trio
from holehe.core import import_submodules, launch_module
from holehe.instruments import TrioProgress

async def controlled_scan(email, sites, max_concurrent=50, timeout=10):
    modules = import_submodules("holehe.modules")
    funcs = [
        modules[m].__dict__[m.split(".")[-1]]
        for m in modules
        if m.split(".")[-1] in sites
    ]
    
    client = httpx.AsyncClient(
        timeout=timeout,
        limits=httpx.Limits(max_connections=max_concurrent)
    )
    
    out = []
    instrument = TrioProgress(len(funcs))
    trio.lowlevel.add_instrument(instrument)
    
    # Use semaphore for controlled concurrency

    sem = trio.Semaphore(max_concurrent)
    
    async def bounded_launch(fn):
        async with sem:
            await launch_module(fn, email, client, out)
    
    async with trio.open_nursery() as n:
        for fn in funcs:
            n.start_soon(bounded_launch, fn)
    
    trio.lowlevel.remove_instrument(instrument)
    await client.aclose()
    return out

```

## Performance Checklist for 120+ Sites

| Factor | Default | Recommended for 120+ Sites |
|--------|---------|---------------------------|
| HTTPX connections | ~100 | 150-200 if customizing |
| Timeout | 10s | 5-15s based on network |
| Modules loaded | ~300 | Filter to 50-150 relevant |
| Stagger/delay | None | 50-100ms if rate-limited |
| Progress visibility | Clearing | Use `--no-clear` |

## Summary

- **Holehe uses a shared HTTPX async client and Trio nurseries** for concurrent site checking, with architecture defined in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)
- **Connection pool defaults (~100) may bottleneck** 120+ simultaneous requests; customize `httpx.Limits` for higher throughput
- **Every module starts as an immediate coroutine**, so reducing the module count via filtering or `-NP` directly improves performance
- **Timeout tuning (`-T`) balances completeness against speed**—aggressive values suit fast networks, generous values suit VPNs
- **Rate limiting manifests as progress bar stalls**; use `--no-clear` to detect and consider custom delays for throttled targets
- **CPU usage remains minimal**; focus optimizations on I/O concurrency and network behavior

## Frequently Asked Questions

### How many concurrent connections does Holehe use by default?

Holehe relies on HTTPX's default connection limits, approximately **100 concurrent keepalive connections**. For 120+ sites, this creates mild queuing. The shared client is instantiated at `core.py:L13-L14` without explicit limits, so the final behavior depends on HTTPX's internal defaults for your version.

### Can I limit how many sites run simultaneously?

Holehe does not expose a built-in concurrency limiter. All modules start immediately via `nursery.start_soon()` in `core.py:L18-L21`. To enforce limits, use a `trio.Semaphore` wrapper as shown in the programmatic example above, or reduce the module count through filtering.

### Why does Holehe slow down after 100-150 sites?

The progress bar often stalls when **rate limits trigger across multiple targets**. Each 429 response consumes a connection slot and delays result collection. Check for this pattern with `--no-clear` and consider targeting fewer sites or adding inter-request delays.

### Is Holehe suitable for scanning thousands of email addresses?

Holehe is optimized for **breadth (many sites) over volume (many emails)**. For large email lists, wrap Holehe calls with your own orchestration that respects remote rate limits, or implement backoff strategies in custom modules to avoid IP bans.