# How to Configure CDP Mode for Stealth Browsing in MediaCrawler

> Learn how to configure CDP mode for stealth browsing in MediaCrawler. This guide explains connecting to a remote debugging endpoint and injecting anti-fingerprinting scripts for enhanced privacy.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-27

---

**MediaCrawler supports stealth browsing via Chrome DevTools Protocol (CDP) by connecting to a remote debugging endpoint and injecting anti-fingerprinting scripts from [`libs/stealth.min.js`](https://github.com/NanmiCoder/MediaCrawler/blob/main/libs/stealth.min.js) into every page context.**

MediaCrawler is an open-source scraping framework that leverages Playwright to automate browser interactions. When you configure CDP mode for stealth browsing in MediaCrawler, the tool connects directly to Chrome or Edge via the Chrome DevTools Protocol, bypassing standard automation detection while maintaining full control over the browser instance.

## Enable CDP Mode in Configuration

All CDP behavior is controlled through the configuration module defined in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py). The following flags determine how the browser launches and maintains its connection.

- **`CDP_CONNECT_EXISTING`**: Set to `True` to attach to an already-running Chrome instance with `--remote-debugging-port` enabled, or `False` to let MediaCrawler launch a fresh browser process.
- **`CDP_DEBUG_PORT`**: The TCP port (default `9222`) where the browser’s CDP server listens.
- **`CUSTOM_BROWSER_PATH`**: Absolute path to a Chrome/Edge binary; if empty, the launcher auto-detects installed browsers.
- **`SAVE_LOGIN_STATE`**: When `True`, creates a persistent user-data directory so cookies and local storage survive restarts.
- **`BROWSER_LAUNCH_TIMEOUT`**: Seconds to wait for the browser to become reachable (default `30`).
- **`AUTO_CLOSE_BROWSER`**: Whether to terminate the browser process on script exit (`True`) or leave it running (`False`).

These values are read in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py), where the decision to attach versus launch is made on lines [108-110](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L108-L110).

## Launch or Attach to a CDP Browser Instance

The `CDPBrowserManager` class in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) handles the lifecycle of the CDP connection. Use the `launch_and_connect()` method to establish the session according to your configuration.

### Launching a Fresh Browser Instance

When `CDP_CONNECT_EXISTING` is `False`, MediaCrawler spawns a new browser process and waits for the CDP endpoint.

```python
from tools.cdp_browser import CDPBrowserManager
from playwright.async_api import async_playwright

async def start_fresh():
    async with async_playwright() as pw:
        manager = CDPBrowserManager()
        context = await manager.launch_and_connect(
            playwright=pw,
            playwright_proxy=None,
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36...",
            headless=False,  # Set True in production

        )
        # Connection established (see lines [97-104](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L97-L104))

        return manager, context

```

### Attaching to an Existing Chrome Process

Set `CDP_CONNECT_EXISTING` to `True` and start Chrome manually with the remote debugging flag:

```bash
chrome --remote-debugging-port=9222

```

Then connect via Python:

```python
async def attach_existing():
    async with async_playwright() as pw:
        manager = CDPBrowserManager()
        # In practice, set this via config or before calling launch_and_connect

        manager.manager_config = {"CDP_CONNECT_EXISTING": True}
        context = await manager.launch_and_connect(playwright=pw, headless=False)
        # Manager waits for endpoint (see lines [166-176](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L166-L176))

        return manager, context

```

## Inject the Stealth Script

After establishing the context, inject the anti-detection logic. MediaCrawler ships with a pre-minified stealth script located at [`libs/stealth.min.js`](https://github.com/NanmiCoder/MediaCrawler/blob/main/libs/stealth.min.js).

Call `add_stealth_script()` to register the script as an init script for every new page:

```python
await manager.add_stealth_script()  # Uses default libs/stealth.min.js

# Or specify a custom path:

# await manager.add_stealth_script("custom_stealth.js")

```

This method verifies the file exists and registers it via `page.addInitScript` so it executes before any page scripts run (see lines [400-407](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L400-L407)).

## Complete Working Example

Combine the configuration, launch, and stealth injection into a single workflow:

```python
import asyncio
from tools.cdp_browser import CDPBrowserManager
from playwright.async_api import async_playwright

async def run_stealth_crawl():
    async with async_playwright() as pw:
        mgr = CDPBrowserManager()
        
        # Launch CDP browser (respects settings in base_config.py)

        ctx = await mgr.launch_and_connect(
            playwright=pw,
            headless=False,
        )
        
        # Enable stealth browsing

        await mgr.add_stealth_script()
        
        # Begin crawling

        page = await ctx.new_page()
        await page.goto("https://www.zhihu.com")
        
        # Cleanup

        await mgr.cleanup()

asyncio.run(run_stealth_crawl())

```

## Persist Browser State Across Sessions

When `SAVE_LOGIN_STATE` is enabled, the manager creates a directory at `browser_data/cdp_<platform>` (see lines [55-63](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L55-L63)). This folder stores cookies, local storage, and extension data, allowing you to maintain login sessions across crawler restarts.

```python

# Ensure in config/base_config.py:

# SAVE_LOGIN_STATE = True

# The manager automatically uses this path when creating the context

ctx = await mgr.launch_and_connect(playwright=pw)

```

## Summary

- **Configuration**: Set `CDP_CONNECT_EXISTING`, `CDP_DEBUG_PORT`, and related flags in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) to control launch behavior.
- **Connection**: Use `CDPBrowserManager.launch_and_connect()` in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) to attach to or spawn a CDP-enabled browser.
- **Stealth**: Call `add_stealth_script()` to inject [`libs/stealth.min.js`](https://github.com/NanmiCoder/MediaCrawler/blob/main/libs/stealth.min.js) and mask automation fingerprints.
- **Persistence**: Enable `SAVE_LOGIN_STATE` to retain cookies in `browser_data/cdp_<platform>` for long-running crawls.

## Frequently Asked Questions

### What is CDP mode in MediaCrawler?

CDP mode drives Chrome or Edge via the Chrome DevTools Protocol instead of standard Playwright browser management. This allows MediaCrawler to attach to existing browser instances, use custom user-data directories, and inject stealth scripts at the CDP level to evade bot detection.

### How does the stealth script prevent detection?

The script in [`libs/stealth.min.js`](https://github.com/NanmiCoder/MediaCrawler/blob/main/libs/stealth.min.js) overrides JavaScript properties that automation frameworks typically expose—such as `navigator.webdriver`, `window.chrome`, and plugin inconsistencies. By executing as an init script (lines [400-407](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py#L400-L407)), it sanitizes the browser environment before any page scripts run.

### Can I use a custom stealth script instead of the default?

Yes. Pass the absolute path to your custom script file when calling `await manager.add_stealth_script("path/to/custom.js")`. The method validates the file existence and registers it as the init script for all subsequent pages.

### Why should I use CDP mode instead of standard Playwright?

CDP mode provides lower-level control over the browser process, enables connection to already-running instances (useful for debugging), and supports persistent user-data directories that standard Playwright contexts may handle differently. It also facilitates stealth configurations that are harder to achieve through high-level browser APIs alone.