# How to Use CDP Mode in MediaCrawler to Connect to an Existing Chrome Browser

> Learn how to use MediaCrawler's CDP Mode to connect to an existing Chrome browser. Preserve sessions and skip launch time with the Chrome DevTools Protocol.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-29

---

**MediaCrawler’s CDP Mode allows you to attach crawlers to an already-running Chrome instance via the Chrome DevTools Protocol, preserving logged-in sessions and avoiding launch overhead.**

You can use **CDP Mode in MediaCrawler** to bypass the standard Playwright browser launch and instead connect to an existing Chrome window. This approach retains cookies, extensions, and authentication states, significantly reducing the risk of anti-bot detection. The implementation centers on the `CDPBrowserManager` class in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py), which handles detection, WebSocket negotiation, and context management.

## Prerequisites: Enable Remote Debugging in Chrome

Before MediaCrawler can attach to your browser, you must enable remote debugging.

1. Open Chrome and navigate to `chrome://inspect/#remote-debugging`
2. Check the option to allow remote debugging for the current browser instance
3. Verify the server is running at `127.0.0.1:9222` (or your custom port)

You can verify the port is accessible using a simple socket probe:

```bash
python - <<'PY'
import socket
port = 9222
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(5)
result = s.connect_ex(('localhost', port))
print('Port open' if result == 0 else 'Port closed')
PY

```

## Configuration Settings

Connection behavior is controlled through [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py). Two flags determine whether MediaCrawler launches a new browser or attaches to an existing one:

- **`CDP_CONNECT_EXISTING`**: Set to `True` to enable attachment mode
- **`CDP_DEBUG_PORT`**: Specifies the port (default: `9222`)

```python

# config/base_config.py

CDP_CONNECT_EXISTING = True
CDP_DEBUG_PORT = 9222

```

## How CDP Browser Management Works

The `CDPBrowserManager` class in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) orchestrates the connection workflow when `CDP_CONNECT_EXISTING` is enabled.

### Port Detection and Waiting

When `launch_and_connect()` is invoked, the manager calls `_connect_existing_browser()`. This method repeatedly tests the debug port using `_test_cdp_connection()`, which performs a socket probe to verify Chrome is listening. If the connection fails initially, the system logs progress and prompts you to enable remote debugging via the Chrome inspect interface.

### WebSocket URL Resolution

If the direct CDP connection fails, the manager falls back to `_get_browser_websocket_url()`. This method queries the `/json/version` HTTP endpoint on the debug port to retrieve the precise WebSocket URL required for Playwright attachment.

### Browser Context Creation

Once connected, `_create_browser_context()` either reuses the first existing browser context (if Chrome already has tabs open) or spawns a fresh context with sensible defaults. Fresh contexts are configured with a 1920×1080 viewport and enabled downloads, ensuring compatibility with MediaCrawler’s scraping requirements.

## Step-by-Step Implementation

### Enable Remote Debugging in Chrome

Launch Chrome with remote debugging enabled, or use the `chrome://inspect` method described above. Ensure no firewall rules block port `9222`.

### Configure MediaCrawler

Edit [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) to set:

```python
CDP_CONNECT_EXISTING = True
CDP_DEBUG_PORT = 9222  # Change if you used a custom startup flag

```

Alternatively, set environment variables if your deployment prefers external configuration.

### Run the Crawler

Execute your crawl command normally. The `CDPBrowserManager` automatically detects the `CDP_CONNECT_EXISTING` flag and attaches to your Chrome instance rather than launching a new browser:

```bash
uv run main.py --platform xhs --lt qrcode --type search

```

The CLI entry point in [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py) parses arguments and instantiates `CDPBrowserManager`, which handles the underlying connection logic transparently.

## Advanced Usage

### Stealth Scripts and Session Management

Once connected, `CDPBrowserManager` provides methods to inject anti-detection scripts and manage cookies:

- **`add_stealth_script()`**: Injects JavaScript to mask automation fingerprints
- **`add_cookies()`**: Seeds the browser context with session cookies
- **`get_cookies()`**: Extracts current cookies for persistence

These methods operate on the attached context, allowing you to maintain state across crawl sessions without re-authentication.

### Programmatic Access

You can also instantiate the manager directly for custom workflows:

```python
from tools.cdp_browser import CDPBrowserManager
from playwright.async_api import async_playwright

async def run():
    async with async_playwright() as pw:
        manager = CDPBrowserManager()
        context = await manager.launch_and_connect(
            playwright=pw,
            playwright_proxy=None,
            user_agent=None,
            headless=False,
        )
        page = await context.new_page()
        await page.goto('https://www.xiaohongshu.com')
        # Proceed with crawler logic

```

### Cleanup Behavior

The `cleanup()` method ensures graceful shutdown. When `CDP_CONNECT_EXISTING` is `True`, it skips process termination to avoid killing your existing Chrome instance, closing only the contexts and connections managed by MediaCrawler.

## Summary

- **CDP Mode** in MediaCrawler connects to existing Chrome instances via the Chrome DevTools Protocol, preserving login states and reducing detection risks.
- Configuration is controlled by `CDP_CONNECT_EXISTING` and `CDP_DEBUG_PORT` in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py).
- The `CDPBrowserManager` class in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) handles port detection, WebSocket resolution, and context creation.
- Use `chrome://inspect/#remote-debugging` to enable remote debugging on port `9222` before running crawlers.
- Run commands normally via [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py); the manager automatically attaches to the existing browser when configured.
- Cleanup routines preserve the external Chrome process while closing managed resources.

## Frequently Asked Questions

### What is CDP Mode in MediaCrawler?

CDP Mode allows MediaCrawler to connect to an existing Chrome browser instance using the Chrome DevTools Protocol rather than launching a fresh Playwright-managed browser. This preserves the browser’s cookies, extensions, and logged-in sessions, making crawls faster and less likely to trigger anti-bot measures.

### How do I verify my Chrome browser is accepting CDP connections?

You can verify the debug port is accessible by running a socket connection test to `localhost:9222` (or your configured port). In [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py), the `_test_cdp_connection()` method performs this check. If the port is closed, ensure you enabled remote debugging via `chrome://inspect/#remote-debugging` and that Chrome was started with the `--remote-debugging-port` flag if necessary.

### Can I use CDP Mode with browsers other than Chrome?

Yes, the implementation supports any Chromium-based browser that implements the Chrome DevTools Protocol, including Microsoft Edge. The `CDPBrowserManager` uses standard CDP endpoints (`/json/version` for WebSocket discovery) that are compatible with any Chromium derivative running with remote debugging enabled.

### Why should I use CDP Mode instead of launching a new browser?

Using CDP Mode eliminates the overhead of browser startup and allows you to reuse existing authentication states, cookies, and browser profiles. According to the source code in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py), this approach greatly reduces the chance of being blocked by target platforms because the browser fingerprint matches a regular user’s long-running session rather than a fresh automation profile.