# How Scrapling Bypasses Cloudflare Turnstile Using StealthyFetcher

> Learn how Scrapling bypasses Cloudflare Turnstile. StealthyFetcher uses Playwright to analyze challenges, simulate human interaction, and ensure network stability for seamless scraping.

- Repository: [Karim shoair/Scrapling](https://github.com/D4Vinci/Scrapling)
- Tags: how-to-guide
- Published: 2026-03-08

---

**Scrapling's StealthyFetcher automatically detects and solves Cloudflare Turnstile challenges by launching a stealth Playwright browser session that analyzes challenge types, simulates human-like clicks, and waits for network stability before returning the final page response.**

Scrapling is an open-source web scraping library that provides advanced browser automation capabilities. When you need to **bypass Cloudflare Turnstile using StealthyFetcher**, the library orchestrates a sophisticated detection and solving pipeline that handles both non-interactive waiting challenges and interactive click-based verification seamlessly.

## How StealthyFetcher Detects and Solves Cloudflare Challenges

### The Entry Point: StealthyFetcher.fetch and the solve_cloudflare Flag

The bypass mechanism starts in [`scrapling/fetchers/stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/fetchers/stealth_chrome.py), where `StealthyFetcher.fetch` accepts a `solve_cloudflare=True` parameter. This flag is forwarded to `StealthySession` in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py), triggering the automated solver pipeline immediately after page navigation.

Inside `StealthySession.fetch`, the flag is read from validated parameters (`params.solve_cloudflare`). If true, **`_cloudflare_solver(page)`** is invoked after the initial navigation to handle any detected challenges.

### Challenge Detection with Regex Pattern Matching

Before solving, the system must identify the challenge type. The `_detect_cloudflare` helper uses the `__CF_PATTERN__` regex defined in [`_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/_stealth.py) to scan page content and classify challenges as `"non-interactive"`, `"embedded"`, or `"interactive"`. This classification determines the subsequent solving strategy.

The detection logic examines the page content and iframe structure to distinguish between simple waiting pages and complex Turnstile widgets that require user interaction.

### The Solving Algorithm: From Network Idle to Click Simulation

The `_cloudflare_solver` method implements a deterministic resolution flow:

1. **Network Stabilization**: Waits for network idle using `_wait_for_networkidle(page, timeout=5000)` to ensure the challenge fully loads.
2. **Type Determination**: Calls `_detect_cloudflare` to identify the specific challenge variant.
3. **Non-Interactive Handling**: For "Just a moment..." pages, loops with `page.wait_for_timeout(1000)` until the challenge title disappears.
4. **Interactive/Embedded Resolution**: Locates the Turnstile iframe using `page.frame(url=__CF_PATTERN__)`, calculates a random offset within its bounding box, and simulates a human-like click.
5. **Retry Logic**: If the challenge persists after maximum attempts, the solver recurses via `return self._cloudflare_solver(page)`.
6. **Final Validation**: Ensures page stability with `_wait_for_page_stability` before returning the response.

All steps are mirrored in the asynchronous version (`AsyncStealthySession._cloudflare_solver`) with `await`-ed Playwright calls, ensuring identical solving capability for async fetches.

## Implementation Details of the Cloudflare Solver

The solver operates through specific method calls in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py):

- **`_wait_for_networkidle`**: Ensures all network activity ceases before interaction, preventing premature clicks on loading elements.
- **`_detect_cloudflare`**: Analyzes the DOM for Cloudflare-specific markers and iframe patterns.
- **Iframe Targeting**: Uses `page.frame(url=__CF_PATTERN__)` to isolate the Turnstile widget from the parent page context.
- **Human Simulation**: Calculates random coordinates within the challenge bounding box to avoid detection as automated input.

## Code Examples

### Synchronous Fetch with Cloudflare Bypass

```python
from scrapling.fetchers.stealth_chrome import StealthyFetcher

# Automatically solves Cloudflare Turnstile challenges

response = StealthyFetcher.fetch(
    "https://protected-site.example.com",
    solve_cloudflare=True,          # Enable the automated solver

    headless=False,                 # Optional: visualize the solving process

    timeout=30000,                  # Request timeout in milliseconds

)

print(response.status)   # 200

print(response.text)     # HTML content after challenge resolution

```

### Asynchronous Fetch with Cloudflare Bypass

```python
import asyncio
from scrapling.fetchers.stealth_chrome import StealthyFetcher

async def main():
    response = await StealthyFetcher.async_fetch(
        "https://protected-site.example.com",
        solve_cloudflare=True,
        headless=True,
        timeout=30000,
    )
    print(response.status)
    print(response.text)

asyncio.run(main())

```

### Custom Page Actions After Challenge Resolution

```python
def scroll_page(page):
    # Execute custom automation after Cloudflare is solved

    page.evaluate("window.scrollTo(0, document.body.scrollHeight)")

response = StealthyFetcher.fetch(
    "https://protected-site.example.com",
    solve_cloudflare=True,
    page_action=scroll_page,  # Runs after challenge resolution

)

```

## Key Source Files and Architecture

| File | Role in Cloudflare Bypass |
|------|---------------------------|
| [`scrapling/fetchers/stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/fetchers/stealth_chrome.py) | Defines `StealthyFetcher` and forwards the `solve_cloudflare` flag to the session layer. |
| [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py) | Implements `StealthySession` and `AsyncStealthySession`, containing the core `_cloudflare_solver`, `_detect_cloudflare`, and challenge interaction logic. |
| [`scrapling/engines/_browsers/_validators.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_validators.py) | Validates the `solve_cloudflare` parameter and other session configuration options. |
| [`scrapling/engines/toolbelt/convertor.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/toolbelt/convertor.py) | Constructs the final `Response` object returned after successful challenge resolution. |

These files together give Scrapling the ability to **detect**, **interact with**, and **solve** Cloudflare Turnstile challenges automatically, providing a transparent, "stealthy" fetch experience for end users.

## Summary

- **Scrapling bypasses Cloudflare Turnstile** through the `StealthyFetcher` class by launching a stealth Playwright browser session.
- The **`solve_cloudflare=True`** parameter triggers an automated pipeline that detects challenge types using regex pattern matching against `__CF_PATTERN__`.
- The solver handles **non-interactive** challenges by waiting for page redirects and **interactive/embedded** challenges by simulating human-like clicks within Turnstile iframes.
- Implementation resides primarily in [`scrapling/engines/_browsers/_stealth.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/engines/_browsers/_stealth.py), with entry points in [`scrapling/fetchers/stealth_chrome.py`](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/fetchers/stealth_chrome.py).
- Both synchronous and asynchronous fetch methods support Cloudflare bypass with identical configuration parameters.

## Frequently Asked Questions

### What types of Cloudflare challenges can StealthyFetcher solve?

Scrapling's StealthyFetcher can solve three types of Cloudflare challenges: **non-interactive** (the "Just a moment..." waiting page), **embedded** Turnstile widgets, and **interactive** challenges requiring explicit user verification. The solver automatically detects the challenge type using the `__CF_PATTERN__` regex and applies the appropriate resolution strategy for each variant.

### Is headless mode supported when bypassing Cloudflare Turnstile?

Yes, headless mode is fully supported, though running with `headless=False` can be useful for debugging. The Cloudflare solver operates entirely within the Playwright browser context, so it functions regardless of whether the browser window is visible. However, some highly protected sites may employ additional bot detection that correlates with headless indicators, so visible mode occasionally provides better success rates.

### How does the retry mechanism work if the challenge fails initially?

The `_cloudflare_solver` method implements recursive retry logic. If the challenge persists after the maximum number of attempts (clicks or waits), the solver calls itself recursively via `return self._cloudflare_solver(page)`. This creates a robust retry loop that continues until the challenge clears or the overall request timeout is reached, ensuring high reliability against intermittent network delays or slow challenge rotations.

### Can I execute custom actions after Cloudflare is solved but before returning the response?

Yes, the `page_action` parameter accepts a callable function that executes immediately after the Cloudflare challenge is resolved. This function receives the Playwright `page` object as its argument, allowing you to scroll, click additional elements, or execute JavaScript before Scrapling constructs and returns the final `Response` object. This is implemented in the session's fetch logic after the `_cloudflare_solver` completes successfully.