# How to Handle Verification Challenges in MediaCrawler: Slider Captchas and SMS Verification

> Learn how to handle verification challenges in MediaCrawler. Automate slider captchas with image processing and SMS verification via FastAPI webhooks for seamless operation.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-27

---

**MediaCrawler handles verification challenges by automating slider captcha solving through image processing and human-like mouse movements, while SMS verification is managed via a FastAPI webhook that caches six-digit codes for automatic form submission.**

The NanmiCoder/MediaCrawler repository provides a robust Python framework for scraping Chinese social media platforms like Douyin (TikTok China). To handle verification challenges in MediaCrawler effectively, you must understand its dual approach: programmatic image analysis for slider captchas and asynchronous webhook processing for SMS verification codes.

## Understanding MediaCrawler's Verification Challenge Architecture

MediaCrawler faces two primary verification mechanisms common to Chinese social media platforms. The first involves **slider captchas** that require image recognition and simulated human interaction. The second involves **SMS verification codes** sent to mobile numbers during login flows. The architecture separates these concerns into dedicated utility modules while maintaining a unified login orchestration layer in the platform-specific login handlers.

## Handling Slider Captchas (Image Verification)

Slider captchas appear when Douyin detects automated behavior. The crawler solves these challenges by analyzing the visual gap between images and simulating realistic mouse movements.

### Detecting the Captcha Challenge

The verification detection begins in [`media_platform/douyin/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/douyin/login.py). The `check_page_display_slider` method waits for the CSS selector `#captcha-verify-image` to appear on the page. If the element is not found within 30 seconds, the function returns `False`, indicating no verification is required. When detected, the method initiates the solving workflow by calling `move_slider` with configurable difficulty parameters.

```python

# From media_platform/douyin/login.py (lines 71-80)

async def check_page_display_slider(self, move_step: int = 5, slider_level: str = "easy"):
    # Wait for captcha element

    # If found, proceed to move_slider

    await self.move_slider(move_step, slider_level)

```

### Image Processing and Gap Detection

Once detected, the crawler extracts the background and gap images from the DOM. In `move_slider` (lines 223-236), the code retrieves the `src` attributes of both the background image and the gap image using the XPath `//*[@id="captcha_container"]/div/div[2]/img[2]`. These URLs are passed to the `Slide` class in [`tools/slider_util.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/slider_util.py).

The `Slide.discern()` method executes a three-step image processing pipeline:

1. **Noise reduction**: `clear_white` removes white noise from the images
2. **Edge detection**: `cv2.Canny` identifies edges in the processed images
3. **Template matching**: `cv2.matchTemplate` locates the gap position by comparing the gap image against the background

The method returns the x-coordinate of the matched gap, which determines the required slide distance in pixels.

```python

# From tools/slider_util.py (lines 30-43)

def discern(self):
    # Download and resize images to temp_image/

    # Apply edge detection and template matching

    # Return the x-coordinate offset

```

### Generating Human-Like Movement Tracks

After calculating the distance, MediaCrawler generates a movement track that mimics human acceleration and deceleration patterns. The `utils.get_tracks(distance, slider_level)` function in [`tools/slider_util.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/slider_util.py) (lines 45-84) supports two difficulty levels:

- **Easy**: Uses `get_track_simple` to create basic acceleration/deceleration curves
- **Hard**: Leverages the `easing` module to produce sophisticated exponential ease-out curves that replicate genuine user behavior more accurately

The hard level implementation in [`tools/easing.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/easing.py) significantly reduces detection rates by avoiding linear or predictable movement patterns.

### Executing the Slider Drag

With the track calculated, `DouYinLogin.move_slider` grabs the gap element using Playwright, computes its center coordinates, and iterates through the generated track points. The method uses `mouse.move` to drag the slider across each increment, then releases the mouse button to complete the verification.

```python

# From media_platform/douyin/login.py (lines 56-64)

async def move_slider(self, move_step: int, slider_level: str):
    gap_element = await self.context_page.locator('xpath=...').first
    # Calculate center and execute track

    for track in tracks:
        await self.context_page.mouse.move(track_x, track_y)

```

### Retry Logic and Failure Handling

If the platform responds with "操作过慢" (operation too slow) or "提示重新操作" (please retry), the code automatically clicks the refresh link and retries up to 20 times. This resilience is implemented in [`media_platform/douyin/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/douyin/login.py) (lines 94-108), ensuring the crawler can recover from transient validation failures without manual intervention.

## Handling SMS Verification Challenges

SMS verification requires external coordination to receive codes sent to mobile numbers during login.

### Receiving SMS via FastAPI Webhook

MediaCrawler runs a lightweight FastAPI server in [`recv_sms.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/recv_sms.py) (lines 55-63) that accepts POST payloads containing SMS text from external SMS forwarding services. This decouples the crawler from direct mobile network access while enabling real-time code receipt.

```python

# From recv_sms.py

@app.post("/")
async def receive_sms_notification(request: Request):
    # Receive JSON payload with platform, number, and content

    # Extract and cache the verification code

```

### Extracting and Caching Verification Codes

The `extract_verification_code` function uses the regular expression `\b[0-9]{6}\b` to isolate the first six-digit token from the message text. The extracted code is stored in the configured cache (Redis by default) with a **3-minute TTL** under the key format `platform_number` (e.g., `dy_13800138000`). This implementation appears in [`recv_sms.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/recv_sms.py) (lines 46-53 and 73-78).

### Polling and Auto-Submission

During mobile login (`DouYinLogin.login_by_mobile`), the crawler polls the cache every second for up to 2 minutes (`max_get_sms_code_time`). Once the code appears in the cache, the automation fills the verification input field and clicks the submit button. A TODO comment in the source code (line 68) notes that developers should validate the submission result, though the current implementation trusts the cached value.

```python

# From media_platform/douyin/login.py (lines 53-68)

while time.time() - start_time < max_get_sms_code_time:
    sms_code = await self.cache.get(f"dy_{self.login_phone}")
    if sms_code:
        await self.context_page.locator("input...").fill(sms_code)
        await self.context_page.locator("button...").click()

```

## Configuration and Customization

Several configuration options control verification challenge handling:

- **Login Type**: Set `config.LOGIN_TYPE` to `qrcode`, `phone`, or `cookie` in `DouYinLogin.__init__` to determine whether SMS or slider-only paths are activated
- **CDP Mode**: `ENABLE_CDP_MODE` in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) toggles between Chrome DevTools Protocol and full Playwright mode, affecting image capture reliability for slider detection
- **Slider Difficulty**: Pass `slider_level="hard"` to `check_page_display_slider` for platforms with stricter bot detection

## Code Examples

### Triggering Slider Handling Manually

For testing or custom workflows, you can force the slider verification path:

```python
from tools import utils
from media_platform.douyin.login import DouYinLogin
from playwright.async_api import async_playwright

async def demo_slider():
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context()
        page = await context.new_page()
        await page.goto("https://www.douyin.com/")
        
        login = DouYinLogin(
            login_type="phone",
            browser_context=context,
            context_page=page,
            login_phone="13800138000",
        )
        # Force the slider check with hard difficulty

        await login.check_page_display_slider(move_step=5, slider_level="hard")

```

### Simulating SMS Webhook Delivery

Test the SMS reception flow by posting to the local endpoint:

```python
import httpx

payload = {
    "platform": "douyin",
    "current_number": "13800138000",
    "from_number": "1069421xxx134",
    "sms_content": "【抖音】您的验证码是: 123456，3分钟内有效。",
    "timestamp": "1686720601614"
}

response = httpx.post("http://127.0.0.1:8000/", json=payload)
print(response.json())  # {"status": "ok"}

```

After this webhook, `login_by_mobile` will automatically retrieve code `123456` from the cache and submit it.

### Customizing Motion Curves

Switch to the hard difficulty profile for sophisticated anti-detection:

```python
from tools import utils

distance = 120  # Pixels from Slide.discern()

hard_tracks = utils.get_tracks(distance, level="hard")

# Returns a list of integers with exponential ease-out curve

```

## Summary

- **Slider captchas** are handled through computer vision (OpenCV template matching) and human-like mouse trajectory generation in [`tools/slider_util.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/slider_util.py)
- **SMS verification** relies on an external FastAPI webhook ([`recv_sms.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/recv_sms.py)) that extracts six-digit codes and caches them for polling by the login handler
- **Retry mechanisms** include up to 20 attempts for slider failures and 2-minute polling windows for SMS codes
- **Configuration** via `LOGIN_TYPE` and `ENABLE_CDP_MODE` allows adaptation to different platform requirements and detection sensitivities
- **Modular architecture** separates image processing, track generation, and verification receipt into distinct, testable components

## Frequently Asked Questions

### How does MediaCrawler detect when a slider captcha is present?

MediaCrawler uses `check_page_display_slider` in [`media_platform/douyin/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/douyin/login.py) to poll for the CSS selector `#captcha-verify-image` every 30 seconds. If the element appears, the method immediately initiates the image capture and solving sequence. If the timeout elapses without detection, the crawler proceeds with normal login flows assuming no verification is required.

### What makes the "hard" slider level more effective than the "easy" level?

The hard level uses the `easing` module in [`tools/easing.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/easing.py) to generate exponential ease-out curves that replicate genuine human mouse movements, including micro-adjustments and variable acceleration. The easy level uses simple linear interpolation with basic acceleration, which detection systems can identify as robotic movement patterns. For production deployments against strict anti-bot measures, the hard level significantly reduces ban rates.

### Can MediaCrawler handle SMS verification without manual intervention?

Yes, when configured for phone login (`LOGIN_TYPE="phone"`), MediaCrawler automates SMS handling through the FastAPI webhook in [`recv_sms.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/recv_sms.py). The webhook receives the SMS content, extracts the six-digit code using regex `\b[0-9]{6}\b`, and stores it in Redis with a 3-minute TTL. The `login_by_mobile` method polls this cache every second for up to 2 minutes, automatically filling and submitting the code when available.

### Where does MediaCrawler store the temporary images during slider processing?

The `Slide` class in [`tools/slider_util.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/slider_util.py) downloads the background and gap images to the `temp_image/` directory during the `discern()` execution (lines 39-52). These files are processed using OpenCV for edge detection and template matching, then the gap offset is calculated and returned to the login handler for track generation.