What Is `generic_validate` in kaifcodec/user-scanner? Purpose, Usage & Examples

generic_validate is a core helper function that standardizes HTTP request execution and response processing across username and email scanning modules in the kaifcodec/user-scanner OSINT tool.

The generic_validate function eliminates repetitive request-handling code by providing a single, reusable entry point for platform-specific validation logic. It lives in the repository's central orchestration layer, enabling dozens of scan modules to focus purely on interpreting responses rather than managing connections, proxies, or error handling.

How generic_validate Works

Located in [user_scanner/core/orchestrator.py](https://github.com/kaifcodec/user-scanner/blob/main/user_scanner/core/orchestrator.py#L83-L99) (lines 83-99), generic_validate implements a four-step pattern:

1. Creates the request via the repository-wide make_request wrapper, which applies default headers, proxy configuration, and timeouts.
2. Delegates response analysis to a user-provided callable (func) that inspects the httpx.Response and returns a Result.
3. Normalizes output by attaching the display URL (show_url if provided, otherwise the request URL) to the Result.
4. Handles failures gracefully by catching exceptions and returning Result.error, allowing the scanning engine to continue without crashing.

This design lets scan modules remain small and focused—typically under 10 lines of custom logic.

generic_validate Function Signature

def generic_validate(
    url: str,
    func: Callable[[httpx.Response], Result],
    *,
    show_url: str | None = None,
    follow_redirects: bool = False,
    headers: dict | None = None,
    **request_kwargs
) -> Result

• url — The endpoint to request.
• func — Your processor function that receives the httpx.Response and returns a Result.
• show_url — Optional override for the URL displayed in scan results.
• follow_redirects — Passed through to make_request for redirect handling.
• headers — Additional headers merged with defaults.
• request_kwargs — Extra arguments forwarded to the underlying request.

Practical generic_validate Examples

Status-Code Validation (Discord)

Many social platforms return predictable HTTP status codes. This pattern appears throughout user_scanner/user_scan/social/ modules:

# user_scanner/user_scan/social/discord.py

from user_scanner.core.orchestrator import generic_validate, Result

def validate_discord(user: str) -> Result:
    url = f"https://discord.com/users/{user}"
    
    def process(resp):
        # Discord: 200 = user exists (available), 404 = user not found (taken)

        return Result.available() if resp.status_code == 200 else Result.taken()
    
    return generic_validate(url, process, show_url=url)

Here generic_validate manages the connection while the process function contains only the status-code check.

HTML Content Inspection (Telegram)

For platforms requiring body analysis, the processor examines response.content:

# user_scanner/user_scan/social/telegram.py

from user_scanner.core.orchestrator import generic_validate, Result

def validate_telegram(user: str) -> Result:
    url = f"https://t.me/{user}"
    
    def process(resp):
        # Telegram includes "tgme_page_extra" in DOM when username exists

        marker = b"tgme_page_extra"
        return Result.available() if marker in resp.content else Result.taken()
    
    return generic_validate(url, process, follow_redirects=True, show_url=url)

The follow_redirects=True parameter ensures generic_validate handles Telegram's redirect behavior before the processor runs.

JSON API Validation (StackB)

Email scan modules use the same pattern for REST endpoints:

# user_scanner/email_scan/gaming/stackb.py

from user_scanner.core.orchestrator import generic_validate, Result

def validate_stackb(email: str) -> Result:
    url = f"https://api.stackb.com/v1/users?email={email}"
    
    def process(resp):
        data = resp.json()
        # API returns {"exists": true} for registered emails

        return Result.available() if data.get("exists") else Result.taken()
    
    return generic_validate(
        url, 
        process, 
        headers={"Accept": "application/json"}
    )

generic_validate applies the Accept header and parses JSON only when the processor calls resp.json().

Supporting Components

File	Purpose	Key Elements Used by generic_validate
user_scanner/core/orchestrator.py	Main implementation	generic_validate definition (lines 83-99)
user_scanner/core/helpers.py	Request infrastructure	make_request, default headers, proxy handling
user_scanner/core/result.py	Result type definitions	Result.available(), Result.taken(), Result.error
Module files (e.g., discord.py, telegram.py)	Concrete validators	Import and invoke generic_validate

Why generic_validate Matters for OSINT Scanning

• DRY principle — Eliminates duplicate request code across 50+ platform modules.
• Consistent behavior — All scans inherit the same timeout, retry, and proxy policies.
• Safe failure mode — Network errors become Result.error entries rather than crashes.
• Testable processors — The func parameter accepts any callable, making unit tests trivial.

Summary

• generic_validate is the central request-orchestration helper in kaifcodec/user-scanner, residing in user_scanner/core/orchestrator.py.
• It combines HTTP execution with pluggable response processing via the func callback.
• Scan modules supply minimal processor functions while generic_validate handles networking, headers, proxies, and error catching.
• The function normalizes output by attaching display URLs and wrapping exceptions in Result.error.
• Three common patterns emerge: status-code checks, HTML marker inspection, and JSON API parsing.

Frequently Asked Questions

What does generic_validate return?

generic_validate returns a Result object from user_scanner/core/result.py. Depending on the processor's logic and request outcome, this will be Result.available(), Result.taken(), or Result.error if an exception occurred during the request or processing.

Can I use generic_validate for non-HTTP validations?

No. generic_validate is tightly coupled to HTTP via its dependency on make_request and the httpx.Response type expected by the func parameter. For non-HTTP checks, you would instantiate Result objects directly without this helper.

How does generic_validate handle proxies and timeouts?

It delegates all connection management to make_request in user_scanner/core/helpers.py, which applies repository-wide defaults including proxy configuration from environment variables and a standard timeout policy. Custom request_kwargs can override these on a per-call basis.

Why does generic_validate accept a show_url parameter?

The show_url parameter decouples the request URL from what appears in scan results. This is useful when the actual request URL contains API keys, tracking parameters, or internal routing that should not be exposed to end users.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}