# How Scrapling StealthyFetcher Achieves Browser Fingerprint Spoofing

> Discover how Scrapling StealthyFetcher spoofs browser fingerprints using real Chromium, authentic headers, and anti-detection patches. Learn about their advanced techniques.

- Repository: [Karim shoair/Scrapling](https://github.com/D4Vinci/Scrapling)
- Tags: deep-dive
- Published: 2026-03-08

---

**Scrapling's StealthyFetcher masks automated browser traffic by launching real Chromium instances via Playwright, injecting authentic HTTP headers through browserforge, spoofing Google-search referers, and applying anti-detection patches like canvas noise and WebRTC blocking.**

Scrapling is an open-source web scraping framework by D4Vinci that provides high-level abstractions over browser automation tools. The **Scrapling StealthyFetcher** serves as the primary interface for bypassing bot detection systems, wrapping Playwright (via the `patchright` wrapper) in a simple API that handles fingerprint randomization, proxy rotation, and challenge solving automatically.

## Architecture of the StealthyFetcher System

### The High-Level Facade

The `StealthyFetcher` class in [`scrapling/fetchers/stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/fetchers/stealth_chrome.py) (lines 13-16) acts as a static utility that creates either a `StealthySession` (synchronous) or `AsyncStealthySession` (asynchronous) based on your execution context. This design hides the complexity of browser lifecycle management behind a single method signature:

```python
StealthyFetcher.fetch(url, headless=True, stealthy_headers=True, ...)

```

When invoked, the fetcher builds a selector configuration and enters a managed context with the appropriate session type. The session validates options through `StealthConfig` (defined in [`scrapling/engines/_browsers/_validators.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_validators.py)) before launching any browser processes, ensuring mutually exclusive flags are resolved before execution.

### Session Management and Browser Context

The core engine resides in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py) (lines 27-38), where `StealthySession` and `AsyncStealthySession` handle the actual browser orchestration. These classes:

- Launch Chromium via `sync_playwright` or `async_playwright`
- Connect via Chrome DevTools Protocol (CDP) or launch persistent contexts
- Maintain a pool of reusable pages (`max_pages`) to avoid context-creation overhead
- Merge generated headers with user-supplied `extra_headers`
- Execute optional `page_action` callbacks after navigation

## Browser Fingerprint Spoofing Mechanisms

### Realistic Header Generation with browserforge

In [`scrapling/engines/toolbelt/fingerprints.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/toolbelt/fingerprints.py) (lines 66-86), the `generate_headers` function creates complete browser header sets—including User-Agent, Accept-Encoding, Accept-Language, and Sec-CH-UA headers—using the **browserforge** library. When `stealthy_headers=True` (the default), these headers replace Playwright's default fingerprint, which detection systems often flag as automation traffic.

The fetcher merges these generated headers with any provided via `extra_headers`, with user values taking precedence. This prevents the "default-requests" signature that basic HTTP clients exhibit.

### Referer Spoofing and Google Search Simulation

The `generate_convincing_referer` function in [`scrapling/engines/toolbelt/fingerprints.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/toolbelt/fingerprints.py) (lines 21-46) extracts the target domain using `tld.get_tld` and constructs a realistic Google search URL (`https://www.google.com/search?q=example`). By default, this makes requests appear to originate from organic search results, a pattern common in legitimate browser behavior.

You can disable this behavior by setting `google_search=False` or override it completely by supplying a custom `referer` in `extra_headers`. The logic check occurs in `StealthySession.fetch` (lines 215-220 in [`_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/_stealth.py)).

### Canvas Noise and WebRTC Leak Prevention

The session applies several runtime patches to the Chromium context:

- **Canvas noise**: When `hide_canvas=True`, the engine injects JavaScript that adds subtle noise to HTML5 canvas operations, preventing fingerprinting based on rendering consistency
- **WebRTC blocking**: Setting `block_webrtc=True` disables WebRTC to prevent local IP address leaks that reveal non-residential networks
- **WebGL toggling**: Controls WebGL vendor and renderer strings to match the generated User-Agent profile
- **Resource blocking**: `disable_resources=True` drops images, fonts, and media files to speed up scraping while maintaining the request fingerprint

## Performance and Reliability Features

### Page Pooling for Concurrent Requests

Rather than creating a new browser context for every request, `StealthySession` maintains a `page_pool` (defined in the `__slots__` block) that reuses pages up to the `max_pages` limit. This pool resets state (cookies, localStorage, sessionStorage) between fetches while preserving the warm browser instance, significantly reducing per-request latency.

### Automatic Cloudflare Challenge Solving

When `solve_cloudflare=True`, the engine detects Cloudflare challenge pages and executes the `_cloudflare_solver` method (lines 111-186 in [`_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/_stealth.py)). This method:

1. Detects the "Just a moment..." interstitial page
2. Clicks the turnstile verification canvas element
3. Waits for the challenge to clear and the target content to load
4. Proceeds with extraction only after validation passes

This eliminates the need for external CAPTCHA-solving services for basic Cloudflare protections.

### Proxy Rotation and Retry Logic

The fetcher accepts a `proxy_rotator` parameter that supplies fresh proxy configurations for each attempt. If a request fails due to proxy errors, the logic in `StealthySession.fetch` (lines 22-29 and 69-82) automatically retries with a new proxy from the rotator up to the specified `retries` limit. This maintains consistent browser fingerprints across different IP addresses, reducing blocking due to IP reputation.

## Implementation Examples

### Synchronous Stealth Fetching

Fetch a page with full fingerprint spoofing and resource blocking:

```python
from scrapling.fetchers.stealth_chrome import StealthyFetcher

resp = StealthyFetcher.fetch(
    "https://example.com",
    headless=True,
    disable_resources=True,
    hide_canvas=True,
    block_webrtc=True,
    solve_cloudflare=True,
    extra_flags=["--no-sandbox"]
)

print(resp.status)       # 200

print(resp.headers)      # Realistic browser headers

```

Source: `StealthyFetcher.fetch` implementation in [`stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/stealth_chrome.py) (lines 13-20).

### Asynchronous Usage with Custom Actions

Perform custom JavaScript execution after page load using the async interface:

```python
import asyncio
from scrapling.fetchers.stealth_chrome import AsyncStealthySession

async def scroll_to_bottom(page):
    await page.evaluate("window.scrollTo(0, document.body.scrollHeight)")

async def main():
    async with AsyncStealthySession(
        headless=False,
        extra_headers={"X-Custom": "value"},
        page_action=scroll_to_bottom,
    ) as engine:
        response = await engine.fetch("https://news.ycombinator.com")
        print(response.status, len(response.text))

asyncio.run(main())

```

Source: Async fetch method in [`_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/_stealth.py) (lines 443-463).

### Disabling Automatic Referer Generation

Override the Google-search referer for API endpoints or direct access:

```python
resp = StealthyFetcher.fetch(
    "https://myapi.com/endpoint",
    stealthy_headers=True,
    extra_headers={"referer": "https://myapp.example.com/"},
    google_search=False   # Disable automatic referer generation

)

```

Source: Referer handling logic in [`_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/_stealth.py) (lines 215-220).

### Proxy Rotation with Retries

Route requests through rotating proxies with automatic failover:

```python
resp = StealthyFetcher.fetch(
    "https://blocked-site.com",
    proxy_rotator=my_rotator,   # Scrapling ProxyRotator instance

    retries=5,
    retry_delay=2,
    stealthy_headers=True
)

```

Source: Proxy handling in `StealthySession.fetch` (lines 22-29).

## Summary

- **Scrapling StealthyFetcher** wraps Playwright in [`scrapling/fetchers/stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/fetchers/stealth_chrome.py) to provide a simple, high-level API for stealth browsing.
- **Fingerprint spoofing** relies on browserforge-generated headers in [`scrapling/engines/toolbelt/fingerprints.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/toolbelt/fingerprints.py), Google-search referer simulation, and runtime patches for canvas and WebRTC.
- **Page pooling** in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py) reuses browser pages across requests to improve performance while isolating session state.
- **Cloudflare bypass** is handled natively via `_cloudflare_solver` (lines 111-186), which automates turnstile clicking without external services.
- **Proxy rotation** integrates directly into the retry logic, allowing fresh IP attempts while maintaining consistent browser fingerprints.

## Frequently Asked Questions

### How does Scrapling StealthyFetcher differ from standard Playwright?

Unlike raw Playwright, which uses detectable default flags and consistent fingerprints, Scrapling's implementation generates realistic browser headers via browserforge, spoofs referers as Google search traffic, and applies runtime patches for canvas noise and WebRTC blocking. The `StealthySession` class manages these configurations automatically while adding page pooling and built-in Cloudflare solving.

### Can I use Scrapling StealthyFetcher without headless mode?

Yes. Setting `headless=False` launches a visible Chromium window, which is useful for debugging or sites that detect headless environments. The fetcher applies the same fingerprint spoofing regardless of visibility mode, though non-headless mode often encounters fewer bot checks due to the presence of a real windowing system.

### What causes the "default-requests" fingerprint that StealthyFetcher prevents?

Basic HTTP clients and unpatched Playwright instances send consistent User-Agent strings, ordered headers, and missing browser-specific headers like `Sec-CH-UA`. Detection systems flag these patterns as automation. StealthyFetcher prevents this by using `generate_headers` in [`scrapling/engines/toolbelt/fingerprints.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/toolbelt/fingerprints.py) to produce randomized, authentic Chromium headers that vary between requests.

### How does the Cloudflare solver work without external CAPTCHA services?

The `_cloudflare_solver` method in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py) detects Cloudflare's "Just a moment..." interstitial and programmatically clicks the turnstile checkbox using Playwright's mouse automation. It then waits for the challenge to clear and the target content to load, handling the JavaScript-based verification entirely within the browser context.