# How to Manage Module-Specific Rate Limits and Errors in Large-Scale Holehe Scans

> Effectively manage module-specific rate limits and errors in large-scale Holehe scans. Learn to use the frequent_rate_limit flag and central wrapper for efficient handling.

- Repository: [Palenath/holehe](https://github.com/megadose/holehe)
- Tags: performance
- Published: 2026-08-15

---

**Use the `frequent_rate_limit` module flag, central `launch_module` wrapper, and standardized result dictionary to handle rate limits and errors at scale.**

Holehe is an async OSINT tool that checks email addresses against hundreds of online services simultaneously. When running large-scale scans, you need robust mechanisms to handle **rate limiting** and **module failures** without crashing the entire run. The tool handles this through a three-layer architecture: per-module rate-limit flags, a centralized execution wrapper, and uniform result reporting.

## Understanding the Rate-Limit Architecture

Every Holehe module follows a strict contract. It receives an email address, an HTTP client, and an output list, then appends a **standardized result dictionary**. This design lets the core runner scale to hundreds of services without custom logic for each site.

### The Standard Result Dictionary

All modules return the same keys, enabling consistent downstream processing:

| Key | Purpose |
|-----|---------|
| `name` | Human-readable service name (e.g., `"twitter"`) |
| `domain` | Service domain (e.g., `"twitter.com"`) |
| `rateLimit` | `True` when HTTP 429 or equivalent detected |
| `error` | `True` for unexpected exceptions (network, parsing, etc.) |
| `exists` | `True` if email found on the service |
| `emailrecovery`, `phoneNumber`, `others` | Optional extracted data |

This uniformity lives in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py), which orchestrates module execution and result formatting.

## Layer 1: Per-Module Rate-Limit Flag with `frequent_rate_limit`

Each module declares a **`frequent_rate_limit`** boolean at the top of its file. This flag signals whether the target service enforces aggressive throttling.

```python

# holehe/modules/social_media/twitter.py#L9

frequent_rate_limit = False

```

Services with stricter limits set this to `True`:

```python

# holehe/modules/social_media/wattpad.py

frequent_rate_limit = True  # This service needs back-off between requests

```

The runner uses this flag to inject controlled delays with `await trio.sleep(..)` between calls, preventing hard blocks on sensitive endpoints.

## Layer 2: Central Error Handling in `launch_module`

All modules execute through **`launch_module`** in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) (lines 66-78). This wrapper guarantees that **no unhandled exception crashes the scan**.

```python

# holehe/core.py#L66-L78

async def launch_module(module, email, client, out):
    try:
        await module(email, client, out)
    except Exception:
        name = str(module).split('<function ')[1].split(' ')[0]
        out.append({
            "name": name,
            "domain": data[name],
            "rateLimit": False,
            "error": True,
            "exists": False,
            "emailrecovery": None,
            "phoneNumber": None,
            "others": None,
        })

```

**Key behaviors:**

- Any exception becomes a structured `error=True` entry
- `rateLimit` remains `False` because the failure wasn't a 429 response
- The scan continues with remaining modules

## Layer 3: Visual Result Classification in `print_result`

After execution, **`print_result`** (lines 22-36) color-codes outcomes for immediate operator awareness:

```python

# holehe/core.py#L22-L36

if results["rateLimit"] and args.onlyused == False:
    websiteprint = print_color("[x] " + results["domain"], "yellow", args)
elif "error" in results.keys() and results["error"] and args.onlyused == False:
    websiteprint = print_color("[!] " + results["domain"], "red", args)

```

| Symbol | Color | Meaning |
|--------|-------|---------|
| `[+]` | Green | Email found (`exists=True`) |
| `[-]` | Magenta | Email not found (`exists=False`) |
| `[x]` | Yellow | Rate-limited (`rateLimit=True`) |
| `[!]` | Red | Error occurred (`error=True`) |

## Running Large-Scale Scans: CLI and Programmatic Examples

### Command-Line Usage

```bash
holehe john.doe@example.com

```

Typical output showing all four states:

```

[+] twitter.com      # email found

[-] github.com       # not registered

[x] instagram.com    # HTTP 429 - rate limited

[!] taringa.com      # network timeout / parsing error

```

### Programmatic Rate-Limit Filtering

```python
import asyncio
import httpx
import holehe.core as hc

async def run(email):
    async with httpx.AsyncClient(timeout=10) as client:
        out = []
        # Load all site modules dynamically

        mods = hc.import_submodules('holehe.modules')
        sites = hc.get_functions(mods)
        
        # Execute each module

        for mod in sites:
            await hc.launch_module(mod, email, client, out)

    # Post-process: isolate rate-limited services

    limited = [r for r in out if r['rateLimit']]
    errors = [r for r in out if r['error']]
    
    print("Rate-limited:", [r['domain'] for r in limited])
    print("Errors:", [r['domain'] for r in errors])
    return out

asyncio.run(run("john.doe@example.com"))

```

### Creating a New Module with Custom Rate-Limit Handling

When adding a service, implement explicit 429 detection and set the appropriate flag:

```python

# holehe/modules/social_media/newservice.py

frequent_rate_limit = True  # Service throttles aggressively

async def newservice(email, client, out):
    try:
        r = await client.get(
            f"https://api.newservice.com/users/{email}",
            headers={"User-Agent": "Holehe/1.0"}
        )
        
        # Explicit rate-limit detection

        if r.status_code == 429:
            out.append({
                "name": "newservice",
                "domain": "newservice.com",
                "rateLimit": True,
                "error": False,
                "exists": False,
                "emailrecovery": None,
                "phoneNumber": None,
                "others": None,
            })
            return
            
        # Normal response processing

        data = r.json()
        out.append({
            "name": "newservice",
            "domain": "newservice.com",
            "rateLimit": False,
            "error": False,
            "exists": data.get("found", False),
            "emailrecovery": data.get("recovery_email"),
            "phoneNumber": None,
            "others": None,
        })
        
    except Exception:
        # Re-raise to let launch_module handle as error=True

        raise

```

## Key Source Files for Rate-Limit Management

| File | Critical Function |
|------|-----------------|
| [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) | **`launch_module`** (exception wrapping), **`print_result`** (visual output), module loader |
| [`holehe/modules/social_media/twitter.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/twitter.py) | Reference module showing `frequent_rate_limit=False` and standard result structure |
| [`holehe/modules/social_media/wattpad.py`](https://github.com/megadose/holehe/blob/main/holehe/modules/social_media/wattpad.py) | Example with `frequent_rate_limit=True` for throttle-sensitive service |
| [`holehe/instruments.py`](https://github.com/megadose/holehe/blob/main/holehe/instruments.py) | Trio progress instruments for async feedback during rate-limit sleeps |

## Scaling Strategies for Production Use

When managing **module-specific rate limits and errors** across thousands of targets:

- **Group modules by `frequent_rate_limit` status** to apply tiered concurrency limits
- **Capture `[x]` and `[!]` outputs** for retry queues with exponential back-off
- **Monitor per-domain error rates** to detect when services change their throttling behavior
- **Use the standardized dictionary** to feed results into SIEM or alerting pipelines

The declarative `frequent_rate_limit` flag lets you tune behavior without modifying core logic—ideal for operational environments where service limits change frequently.

## Summary

- **`frequent_rate_limit`** boolean in each module signals aggressive throttling to the runner
- **`launch_module`** in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) catches all exceptions and converts them to structured `error=True` entries
- **Standard result dictionaries** with `rateLimit` and `error` keys enable uniform processing across hundreds of services
- **`print_result`** color-codes `[x]` (yellow/rate-limited) and `[!]` (red/error) for immediate visual triage
- The architecture scales because new modules only need to set a flag and return the standard keys—no core changes required

## Frequently Asked Questions

### How does Holehe detect rate limits versus other errors?

Holehe relies on **module-level HTTP status code inspection**. When a module receives HTTP 429 (or equivalent), it sets `rateLimit=True` in the result dictionary. The central `launch_module` wrapper only sets `error=True` for unhandled exceptions that bubble up, not for explicitly handled 429 responses. This distinction appears in `print_result` as `[x]` versus `[!]`.

### Can I customize the delay for `frequent_rate_limit=True` modules?

The current implementation in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) uses `trio.sleep` with a fixed short delay. To customize, you would modify the runner loop where it checks `module.frequent_rate_limit` before execution. The modular structure keeps this logic centralized, so per-service tuning only requires changes in one location.

### What happens if a module crashes mid-execution?

The `launch_module` wrapper's `try/except` block guarantees **graceful degradation**. The exception is caught, a result entry with `error=True` is appended, and execution proceeds to the next module. No thrown exception aborts the entire scan, as implemented in [`holehe/core.py`](https://github.com/megadose/holehe/blob/main/holehe/core.py) lines 66-78.

### How do I retry rate-limited services after a cooldown period?

The current Holehe architecture doesn't include automatic retry. For production implementations, filter the output list for `rateLimit=True` entries, then re-run those specific modules after a delay. The standardized dictionary format makes this straightforward to implement in a wrapper script.