# How the kaifcodec/user-scanner Orchestrator Handles Username Scan Batch Running

> Discover how the kaifcodec/user-scanner orchestrator manages username scan batch running with asyncio concurrency, request capping, and configurable filtering. Learn more!

- Repository: [Kaif/user-scanner](https://github.com/kaifcodec/user-scanner)
- Tags: internals
- Published: 2026-09-02

---

**The orchestrator processes username scans in batches using asyncio for concurrency, capping at 60 simultaneous requests, with a Rich progress bar and configurable filtering for loud or NSFW modules.**

The **kaifcodec/user-scanner** repository provides an asynchronous framework for checking username availability across hundreds of sites. Central to this system is the orchestrator in [`user_scanner/core/orchestrator.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/orchestrator.py), which coordinates batch execution while respecting user-defined limits and providing real-time feedback. Understanding how username scan batch running works requires examining its concurrency model, worker architecture, and result aggregation pipeline.

## Core Architecture: The _run_batch Method

The heart of the orchestrator is `_run_batch`, an asynchronous method that accepts:

- `modules`: A list of loaded site modules to execute
- `username`: The target username string
- `config`: A `ScanConfig` dataclass with CLI-derived flags

This method implements **cooperative multitasking** with controlled parallelism to prevent overwhelming target sites or hitting local resource limits.

### Concurrency Control via Semaphore

Before any work begins, `_run_batch` initializes an `asyncio.Semaphore` with `MAX_CONCURRENT_REQUESTS` (default 60) to enforce an upper bound on simultaneous network operations:

```python

# From user_scanner/core/orchestrator.py, lines 90-94

semaphore = asyncio.Semaphore(MAX_CONCURRENT_REQUESTS)

```

This semaphore passes to every worker task, which must acquire it before executing the site's validation function. Users can override the default via the `--concurrency` CLI flag.

### Real-Time Progress Visualization

The orchestrator integrates the **Rich** library for terminal UI feedback:

```python

# Conceptual structure from lines 94-100

with Progress() as progress:
    task = progress.add_task(f"Scanning {username}...", total=len(modules))
    # ... task creation and monitoring

```

The progress bar updates dynamically, showing the site currently being processed and overall completion percentage.

## Worker Execution Model

For each module in the batch, the orchestrator spawns an `asyncio.Task` via `create_task` (lines 107-112). These tasks execute `_async_worker`, a structured coroutine handling individual site checks.

### The _async_worker Lifecycle

Located at lines 63-73 in [`user_scanner/core/orchestrator.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/orchestrator.py), `_async_worker` performs four critical operations:

1. **Metadata extraction** – Retrieves `site_name` and the validation function via helpers from [`user_scanner/core/helpers.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/helpers.py)

2. **Loud module filtering** – Checks `is_loud(module)` and skips execution unless `config.allow_loud` is True. This prevents "noisy" sites that might notify the target user from running unexpectedly

3. **Execution strategy selection** – Determines whether the validation function is a coroutine or synchronous:
   - **Async path**: `await func(username)` for native async validators
   - **Sync path**: `await loop.run_in_executor(_shared_executor, func, username)` for blocking code, using a shared `ThreadPoolExecutor`

4. **Exception normalization** – Catches `asyncio.TimeoutError` and unexpected exceptions, wrapping them in `Result.error` objects with consistent metadata

```python

# Simplified representation of execution path selection

if asyncio.iscoroutinefunction(func):
    result = await func(username)
else:
    result = await loop.run_in_executor(_shared_executor, func, username)

```

## Result Aggregation and Display

As tasks complete, `_run_batch` processes them in completion order using `asyncio.as_completed` (lines 115-127):

- **Category headers** – Prints a grouped header the first time a result from a new category appears (when `show_all` is enabled or the result is visible)
- **Rendering** – Invokes `result.show(configs)` to apply user formatting preferences (verbose mode, color output, etc.)
- **Collection** – Accumulates all `Result` objects into a final list returned to the caller

This streaming approach ensures users see findings immediately rather than waiting for the entire batch to finish.

## High-Level Orchestration Entry Points

The orchestrator exposes three convenience functions wrapping `_run_batch`:

| Function | Purpose | Location |
|----------|---------|----------|
| `run_user_module` | Scan single module or explicit list synchronously via `asyncio.run` | Lines 31-35 |
| `run_user_category` | Load all modules in a category directory and execute batch | Lines 38-45 |
| `run_user_full` | Pre-load all categories with global semaphore, stream results category-by-category | Lines 59-64 |

### Single Module or List Execution

For targeted scans, `run_user_module` handles the event loop management:

```python
from user_scanner.core.orchestrator import run_user_module
from user_scanner.core.helpers import ScanConfig

config = ScanConfig(allow_loud=False, show_all=True)
results = run_user_module("target_user", ["linkedin", "github"], config)

```

### Category-Scoped Scanning

`run_user_category` leverages `load_modules` from helpers to dynamically discover validators:

```python
from user_scanner.core.orchestrator import run_user_category
from user_scanner.core.helpers import ScanConfig
from pathlib import Path

social_path = Path(__file__).parent / "user_scanner" / "user_scan" / "social"
config = ScanConfig(no_nsfw=True)
results = run_user_category(social_path, "developer1", config)

```

### Full Catalog Scanning

`run_user_full` implements the most complex orchestration, maintaining a single semaphore across all categories to honor global concurrency limits while providing category-grouped output.

## Configuration and Filtering

The `ScanConfig` dataclass (defined in [`user_scanner/core/helpers.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/helpers.py)) controls batch behavior:

| Field | Effect on Batch Execution |
|-------|---------------------------|
| `allow_loud` | Permits modules flagged as notification-triggering |
| `no_nsfw` | Excludes adult-content sites from loading |
| `show_all` | Displays negative findings and skipped results |
| `verbose` | Enables detailed error reporting and timing |
| `concurrency` | Overrides `MAX_CONCURRENT_REQUESTS` default |

These flags pass through to `_run_batch` and `_async_worker`, affecting both module selection and result presentation.

## Summary

- The orchestrator's `_run_batch` method in [`user_scanner/core/orchestrator.py`](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/orchestrator.py) implements username scan batch running with **asyncio concurrency** capped at 60 simultaneous requests
- **Semaphore acquisition** ensures respectful scanning that won't overwhelm target infrastructure
- **_async_worker** handles per-site execution with automatic sync/async dispatch and exception normalization
- **Rich progress bars** provide real-time visibility into batch completion status
- **High-level helpers** (`run_user_module`, `run_user_category`, `run_user_full`) offer flexible entry points for different scanning scopes
- `ScanConfig` flags enable fine-grained control over loud modules, NSFW content, and output verbosity

## Frequently Asked Questions

### What limits how many sites the orchestrator scans simultaneously?

The `asyncio.Semaphore` initialized with `MAX_CONCURRENT_REQUESTS` (default 60) controls concurrency. Users can override this via the `--concurrency` CLI flag, which passes through `ScanConfig` to the semaphore constructor.

### How does the orchestrator handle synchronous site validators?

The `_async_worker` function detects synchronous validators using `asyncio.iscoroutinefunction`. Non-async functions execute in a shared `ThreadPoolExecutor` via `loop.run_in_executor()`, preventing blocking the event loop while maintaining uniform async semantics.

### What happens when a site check times out or crashes?

`_async_worker` catches `asyncio.TimeoutError` and general exceptions, wrapping them in `Result.error` objects with site metadata preserved. These error results flow through the normal aggregation pipeline and render according to `config.verbose` settings.

### Can I prevent "noisy" sites from running in a batch scan?

Yes. Set `allow_loud=False` in `ScanConfig` (or omit `--allow-loud` on CLI). The orchestrator checks `is_loud(module)` before executing each validator and skips loud modules unless explicitly permitted.