# How User‑Scanner Performs Email Scanning: A Deep Dive Into the Async Orchestrator Pipeline

> Discover how User-Scanner executes email scanning through its async orchestrator pipeline. Learn about batch and module-specific scans via CLI or API.

- Repository: [Kaif/user-scanner](https://github.com/kaifcodec/user-scanner)
- Tags: deep-dive
- Published: 2026-09-02

---

**User‑Scanner treats an email address as a first‑class scan target and executes it through a dedicated asynchronous pipeline built around the email orchestrator, with support for full‑batch, category‑batch, or module‑specific scans controlled by CLI flags or programmatic API calls.**

The `user-scanner` repository (kaifcodec/user-scanner) is an open‑source OSINT tool designed to probe username and email availability across hundreds of web services. When the target is an **email address** rather than a username, the tool switches to a specialized code path optimized for async I/O, proxy rotation, and modular validation. This article explains exactly how that email scanning pipeline works, from CLI parsing through result aggregation.

---

## CLI Entry Point and Target Loading

Email scanning begins in [`user_scanner/__main__.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/__main__.py). The driver detects the `-e/--email` or `-ef/--email-file` flags at lines 82‑85 and loads the target list at lines 98‑105.

Individual emails are validated with `is_valid_email` (line 57), then expanded through permutations if requested (lines 66‑73). This preprocessing ensures that entries like `alice+tag@example.com` or variant forms are treated as separate scan targets.

```bash

# Scan a single email

user-scanner -e alice@example.com

# Scan multiple emails from a file

user-scanner -ef emails.txt

```

---

## ScanConfig and Global Options

Before dispatching, the driver instantiates a **`ScanConfig`** object (line 89). This dataclass carries:

- `allow_loud` – whether to include noisy modules that may alert the target service
- `show_all` – display negative results (taken/unavailable) alongside hits
- `timeout` and `proxy` settings
- concurrency limits

Concurrency can be overridden via `--concurrency`, which propagates to `set_concurrency` in the email orchestrator (lines 48‑52 of [`__main__.py`](https://github.com/kaifcodec/user-scanner/blob/main/__main__.py)). The default **`MAX_CONCURRENT_REQUESTS`** is **25**, enforced via semaphore to prevent overwhelming target APIs.

```python
from user_scanner.core.helpers import ScanConfig

config = ScanConfig(
    allow_loud=False,
    show_all=True,
    no_nsfw=False,
    verbose=False
)

```

---

## Orchestrator Dispatch: Three Batch Modes

The driver selects one of three orchestrator functions based on user input. All are thin wrappers around [`user_scanner/core/email_orchestrator.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/email_orchestrator.py):

| Mode | Function | When Used | Line in [`__main__.py`](https://github.com/kaifcodec/user-scanner/blob/main/__main__.py) |
|------|----------|-----------|----------------------|
| **Full‑batch** | `run_email_full_batch` | No `-m` or `-c` flags; scan all email modules | 98 |
| **Category‑batch** | `run_email_category_batch` | `-c <category>` flag; scan one directory | 88 |
| **Module‑batch** | `run_email_module_batch` | `-m <module1,module2>` flag; scan specific modules | 63 |

These functions share a common async implementation, differing only in which validator modules they collect.

```bash

# Full scan of all email modules

user-scanner -e bob@domain.com

# Scan only the "professional" category

user-scanner -e bob@domain.com -c professional

# Scan specific modules with higher concurrency

user-scanner -e bob@domain.com -m github,gitlab --concurrency 40

```

---

## The Async Worker: `_async_worker`

The core of email scanning is **`_async_worker`** in [`email_orchestrator.py`](https://github.com/kaifcodec/user-scanner/blob/main/email_orchestrator.py). For each module in the batch, it performs:

1. **Semaphore acquisition** – respects `MAX_CONCURRENT_REQUESTS` (lines 55‑60)
2. **Validator retrieval** – calls `get_scan_func` to fetch the module's `validate_<service>` function (line 74)
3. **Loud module filtering** – skips if `allow_loud=False` and module is marked loud (line 89)
4. **Execution dispatch** – awaits coroutines directly or off‑loads sync functions to a thread pool (lines 95‑98)
5. **Exception handling** – wraps errors or timeouts into `Result.error` objects (lines 99‑102)
6. **Metadata enrichment** – attaches `site_name`, `username`, `category`, and **`is_email=True`** (line 104)

This design allows modules to be written as either `async def` or standard `def` functions without caller concern.

---

## Batch Orchestration with Live Progress

The **`_run_batch`** function manages the overall execution:

- Creates a `rich` progress bar for terminal feedback (lines 19‑26)
- Spawns an `asyncio.Task` per module (lines 33‑44)
- Streams results as they complete rather than waiting for all (lines 47‑58)
- Groups output by category for readable console presentation

Results accumulate into a list that preserves completion order for downstream formatters.

---

## Proxy and Timeout Injection

To ensure uniform network behavior, the orchestrator **monkey‑patches** `httpx.AsyncClient` and `httpx.Client` at lines 23‑52. Every instantiated client automatically receives:

- The user‑provided proxy from `get_proxy()`
- The global timeout from `get_global_timeout()`

This eliminates repetitive boilerplate in individual validation modules and centralizes network policy.

---

## Result Aggregation and Export

After the batch completes, control returns to [`__main__.py`](https://github.com/kaifcodec/user-scanner/blob/main/__main__.py) (lines 190‑225). The driver:

1. Merges results from all orchestrator calls
2. Attaches the original `seed_target` for traceability
3. Dispatches to formatters in [`user_scanner/core/formatter.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/formatter.py)

Supported output formats include **console (default)**, **JSON**, **CSV**, and **PDF**. The `Result` class in [`result.py`](https://github.com/kaifcodec/user-scanner/blob/main/result.py) encapsulates states: `available`, `taken`, `error`, or `skipped`.

```bash

# Export email scan results to JSON

user-scanner -ef emails.txt -f json -o results.json

```

---

## Programmatic API Usage

The orchestrator functions are importable for library use:

```python
from user_scanner.core.email_orchestrator import run_email_full_batch
from user_scanner.core.helpers import ScanConfig

config = ScanConfig(allow_loud=False, show_all=True, no_nsfw=False, verbose=False)
results = run_email_full_batch("charlie@sample.org", config)

for r in results:
    r.show(config)  # formatted console output

```

This enables integration into larger OSINT workflows without subprocess overhead.

---

## Summary

- **Email targets** are detected via `-e`/`--email` or `-ef`/`--email-file`, validated, and optionally permutation‑expanded in [`__main__.py`](https://github.com/kaifcodec/user-scanner/blob/main/__main__.py).
- **`ScanConfig`** centralizes options including the `--concurrency` override that feeds `set_concurrency()` in the orchestrator.
- Three dispatch modes—**full‑batch**, **category‑batch**, **module‑batch**—select which validators run without duplicating async logic.
- **`_async_worker`** handles per‑module execution with semaphore‑controlled concurrency, loud‑module filtering, and uniform error wrapping.
- **Monkey‑patched `httpx` clients** enforce global proxy and timeout settings across all modules.
- **Streaming results** feed into `Result` objects that formatters transform to console, JSON, CSV, or PDF output.

---

## Frequently Asked Questions

### What concurrency limit does user‑scanner use for email scans?

User‑scanner defaults to **25 concurrent requests** (`MAX_CONCURRENT_REQUESTS`), enforced by a semaphore in `_async_worker`. This can be overridden via the `--concurrency` CLI flag, which propagates to `set_concurrency()` in the email orchestrator.

### Can email modules be synchronous, or must they be async?

Both are supported. The `_async_worker` checks whether the retrieved `validate_<service>` function is a coroutine; if so it awaits it directly, otherwise it off‑loads execution to a thread pool. Module authors may choose either style.

### How does user‑scanner handle proxies and timeouts for email validation?

The orchestrator monkey‑patches `httpx.AsyncClient` and `httpx.Client` (lines 23‑52) so every request automatically inherits the user‑provided proxy and global timeout. This centralizes network configuration without requiring per‑module changes.

### What output formats are available for email scan results?

Results can be displayed in the terminal (default) or exported as **JSON**, **CSV**, or **PDF** via the `-f`/`--format` and `-o`/`--output` flags. The [`formatter.py`](https://github.com/kaifcodec/user-scanner/blob/main/formatter.py) module handles all transformations from `Result` objects.