# How Holehe Handles Websites with Frequent Rate Limiting: Architecture and Implementation

> Discover how Holehe tackles frequent rate limiting with its per-module flag and Trio-based dynamic backoff. Maintain speed on permissive sites while automatically throttling aggressive ones.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: architecture
- Published: 2026-08-20

---

**Holehe uses a per-module `frequent_rate_limit` flag combined with Trio-based dynamic backoff to automatically throttle requests to aggressively rate-limited services while maintaining full speed on permissive ones.**

Holehe is an open-source email enumeration tool that queries hundreds of web services to discover account associations. Because many platforms aggressively limit automated traffic, the tool implements a **rate-limit aware architecture** that gracefully degrades performance only where necessary. This article examines how `megadose/holehe` structures its throttling logic, from module-level declarations to centralized execution control.

## The Per-Module Rate Limit Declaration System

Every Holehe module exposes a standardized dictionary containing service metadata. Two critical fields control throttling behavior: `frequent_rate_limit` (boolean) and `rateLimit` (integer, when applicable). This design pushes rate-limit knowledge to the service implementer, who best understands the target platform's behavior.

Modules that commonly trigger rate limits explicitly set `frequent_rate_limit = True`:

- The **BlaBlaCar** transport module declares the flag at lines 9-10 of [`holehe/modules/transport/blablacar.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/transport/blablacar.py)
- The **Wattpad** social-media module uses identical marking at lines 9-10 of [`holehe/modules/social_media/wattpad.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/wattpad.py)

```python

# Typical module structure (from blablacar.py)

from holehe.core import *

async def blablacar(email, client, out):
    # ... request logic ...

    pass

blablacar.frequent_rate_limit = True  # Signals need for throttling

blablacar.rateLimit = 2               # Optional: explicit limit value

```

Modules without aggressive rate limiting omit these flags entirely. The **Twitter** module exemplifies this contrast—it runs at full speed without throttling overhead.

## Centralized Throttling in holehe/core.py

The scanning engine in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) consumes the module list and inspects each module's `frequent_rate_limit` attribute before execution. When the flag is `True`, the core applies **dynamic backoff** using the asynchronous Trio library.

```python

# Simplified conceptual flow based on core.py implementation

import trio

async def execute_module(module, email, client, out):
    if getattr(module, 'frequent_rate_limit', False):
        # Insert controlled delay before request

        await trio.sleep(backoff_duration)
    
    # Proceed with actual request

    return await module(email, client, out)

```

This centralized approach provides three advantages:

- **Consistency** — All rate-limited services receive identical throttling treatment
- **Maintainability** — Throttling logic changes in one location affect all modules
- **Observability** — The core can log and instrument delays uniformly

## Automatic Retry with Exponential Backoff

When a request still returns **HTTP 429** (Too Many Requests), Holehe's core routine catches the exception and escalates the delay:

1. Initial retry waits 2-4 seconds
2. Subsequent retries double the interval (exponential backoff)
3. Maximum retry count is configurable (default: 3 attempts)

This layered defense—proactive throttling plus reactive retry—maximizes result recovery without risking permanent IP blocks.

## Progress Tracking with TrioProgress

The [`holehe/instruments.py`](https://github.com/megadose/holehe/blob/main/holehe/instruments.py) file defines `TrioProgress`, a **Trio-compatible instrument** that monitors request timing across the entire scan. When throttling delays are injected, the instrument logs these pauses, enabling accurate progress bar display in the CLI.

```python

# From instruments.py - TrioProgress integration

class TrioProgress(trio.abc.Instrument):
    def before_io_wait(self, timeout):
        # Tracks actual wait time including injected delays

        self.total_wait += timeout

```

This prevents the user interface from stalling or showing inaccurate completion estimates when scanning rate-limited services.

## Command-Line and Programmatic Usage

Holehe automatically applies all throttling logic when invoked from the command line:

```bash

# Verbose mode shows rate-limit handling in real-time

holehe -v user@example.com

```

For custom integrations, you can inspect module flags and replicate the backoff pattern:

```python
import trio
from importlib import import_module

# Check if a service requires throttling

mod = import_module('holehe.modules.social_media.wattpad')
print(mod.frequent_rate_limit)   # → True

# Manual throttling for custom workflows

async def throttled_check(email, module_name):
    mod = import_module(f'holehe.modules.{module_name}')
    
    if getattr(mod, 'frequent_rate_limit', False):
        await trio.sleep(2)  # Match Holehe's default backoff

    
    # Execute with appropriately configured client...

```

## Implementation Contrast: Rate-Limited vs. Standard Services

| Characteristic | Rate-Limited Modules (BlaBlaCar, Wattpad) | Standard Modules (Twitter) |
|----------------|-------------------------------------------|----------------------------|
| Flag set | `frequent_rate_limit = True` | Flag omitted |
| Pre-request delay | Yes, via `trio.sleep()` | None |
| Retry on 429 | Exponential backoff | Standard retry only |
| Progress tracking | Delay logged by TrioProgress | Standard timing only |
| Scan throughput | Reduced for that module | Full speed |

This differential treatment ensures that a single restrictive service cannot bottleneck the entire enumeration job.

## Summary

- **Per-module flags** (`frequent_rate_limit`, `rateLimit`) declare service-specific rate-limit risk
- **[`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py)** centralizes throttling decisions and applies `trio.sleep()` delays automatically
- **Exponential backoff retry** handles residual 429 responses without manual intervention
- **`TrioProgress` instrument** maintains accurate CLI progress reporting during injected delays
- **Unified module interface** enables single-point rate-limit management across all 100+ services

## Frequently Asked Questions

### How does Holehe know which websites need throttling?

Module authors set `frequent_rate_limit = True` based on observed behavior during development. The flag is defined in the module's global scope, same as `rateLimit`. This crowdsourced knowledge accumulates as contributors test services against real-world rate limiting.

### Can I disable rate-limit handling to speed up scans?

No built-in flag exists to override module declarations. You would need to modify [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) to ignore the `frequent_rate_limit` check, though this risks IP bans and incomplete results. The defaults prioritize reliability over raw speed.

### What happens if a service changes its rate-limit policy?

The modular architecture allows rapid updates. A contributor can adjust `frequent_rate_limit` or `rateLimit` in the specific module file (e.g., [`holehe/modules/transport/blablacar.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/transport/blablacar.py)) without touching core logic. Changes propagate immediately on next scan.

### Does Holehe handle CAPTCHAs that appear after rate limiting?

No—Holehe focuses on **preemptive and reactive throttling** to avoid triggering CAPTCHAs. If a site serves CAPTCHA challenges, the request typically fails and is marked accordingly. The tool does not integrate CAPTCHA-solving services.