# How to Use MediaCrawler CDP Mode with an Existing Chrome Browser

> Connect MediaCrawler to your existing Chrome browser using CDP mode. Reuse sessions and cookies, avoid detection, and speed up your scraping tasks. Learn how now.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-26

---

**MediaCrawler connects to an existing Chrome instance via the Chrome DevTools Protocol (CDP) to reuse logged-in sessions, cookies, and browser extensions, eliminating the overhead of launching fresh browser instances and significantly reducing detection risks.**

MediaCrawler is an open-source crawling framework that supports **CDP mode with existing Chrome** to streamline data extraction from platforms like Xiaohongshu, Douyin, and others. By attaching to a browser you already have running with remote debugging enabled, you preserve authentication states and avoid repetitive login flows. This approach leverages the `CDPBrowserManager` class in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) to manage the connection lifecycle, port detection, and context creation.

## How CDP Mode Works in MediaCrawler

The CDP implementation centers on the `CDPBrowserManager` class located in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py). This orchestrator handles two distinct modes: launching a fresh Playwright-managed browser or attaching to an existing Chrome instance. When configured for existing browser attachment, the manager bypasses the standard launch sequence and instead establishes a WebSocket connection to Chrome's remote debugging endpoint.

The workflow depends on two critical configuration flags in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py):

- **`CDP_CONNECT_EXISTING`**: Boolean flag that switches between launching new browser (False) or connecting to existing (True)
- **`CDP_DEBUG_PORT`**: Integer specifying the port for the CDP connection (default: 9222)

## Prerequisites: Enabling Remote Debugging in Chrome

Before configuring MediaCrawler, you must enable remote debugging on your Chrome instance. This exposes the Chrome DevTools Protocol on a local port, allowing external tools to control the browser.

1. Open Chrome and navigate to `chrome://inspect/#remote-debugging`
2. Enable "Allow remote debugging for this browser instance"
3. Verify the server is running at `127.0.0.1:9222` (or your custom port)

Verify the port is accessible using a simple socket probe:

```python
import socket

def test_cdp_port(port=9222):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(5)
    result = s.connect_ex(('localhost', port))
    print('Port open - CDP ready' if result == 0 else 'Port closed')
    return result == 0

test_cdp_port()

```

## Configuring MediaCrawler for Existing Chrome Connection

Edit [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) to enable CDP mode with your existing browser:

```python

# config/base_config.py

CDP_CONNECT_EXISTING = True
CDP_DEBUG_PORT = 9222  # Change if using custom port

SAVE_LOGIN_STATE = True  # Optional: preserve cookies between sessions

```

Alternatively, set environment variables before running your crawler:

```bash
export CDP_CONNECT_EXISTING=true
export CDP_DEBUG_PORT=9222

```

## The Connection Workflow Explained

When `CDP_CONNECT_EXISTING` is `True`, the `launch_and_connect()` method initiates a specific sequence defined in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py):

### Port Detection and Waiting

The `_test_cdp_connection()` method performs repeated socket probes against the configured debug port. If the connection fails initially, the manager logs progress and prompts you to verify remote debugging is enabled via `chrome://inspect/#remote-debugging`.

### WebSocket URL Resolution

If direct CDP connection fails, the system falls back to `_get_browser_websocket_url()`, which queries the `/json/version` HTTP endpoint on the debug port to retrieve the proper WebSocket URL for Chrome DevTools Protocol communication.

### Browser Context Creation

Once connected, `_create_browser_context()` either reuses the first existing context (if the browser already has tabs open) or creates a fresh context with these defaults:

- Viewport: 1920×1080
- Downloads enabled
- Accepts downloads from any origin

## Managing Sessions, Cookies, and Stealth

The `CDPBrowserManager` provides helper methods to manage session state and evade detection:

- **`add_stealth_script()`**: Injects anti-detection scripts into the browser context
- **`add_cookies()`**: Programmatically inserts cookies into the existing Chrome session
- **`get_cookies()`**: Extracts current cookies for persistence

These methods operate on the browser context obtained from the existing Chrome instance, allowing you to synchronize login states between manual browsing and automated crawling.

## Running the Crawler with Existing Chrome

Execute your crawler using the standard CLI interface while the configuration points to your existing browser:

```bash
uv run main.py --platform xhs --lt qrcode --type search

```

The `CDPBrowserManager` instance created in [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py) automatically detects the `CDP_CONNECT_EXISTING` setting and attaches to your running Chrome rather than spawning a new process.

## Cleanup and Resource Management

The `cleanup()` method in `CDPBrowserManager` handles resource termination intelligently. When attached to an external browser (`CDP_CONNECT_EXISTING = True`), it skips process termination and only closes the Playwright contexts it created, leaving your Chrome instance running for future use.

```python
from tools.cdp_browser import CDPBrowserManager
import asyncio

async def safe_cleanup():
    manager = CDPBrowserManager()
    await manager.cleanup()  # Closes contexts, keeps Chrome running

asyncio.run(safe_cleanup())

```

## Summary

- **Enable remote debugging** in Chrome via `chrome://inspect/#remote-debugging` before starting MediaCrawler
- **Set `CDP_CONNECT_EXISTING = True`** in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) to switch from launch mode to attach mode
- **Use `CDP_DEBUG_PORT`** to specify the remote debugging port (default 9222)
- **Leverage `CDPBrowserManager`** in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) to handle connection logic, context creation, and stealth injection
- **Preserve browser state** by connecting to existing Chrome profiles with saved logins and extensions
- **Cleanup safely** using the `cleanup()` method, which respects external browser processes

## Frequently Asked Questions

### How do I check if Chrome is ready for CDP connection?

Verify the debug port is accessible by testing socket connectivity to `localhost:9222` (or your custom port). The `CDPBrowserManager` includes internal probing via `_test_cdp_connection()`, but you can manually check using the Python socket example provided above or by visiting `http://localhost:9222/json/version` in your browser.

### Can I use Chrome profiles or extensions with CDP mode?

Yes. Connecting to an existing Chrome instance preserves all profile data, extensions, and cookies because MediaCrawler attaches to the running process rather than launching an isolated browser. This is the primary advantage of using `CDP_CONNECT_EXISTING` over standard headless mode.

### What happens if the debug port is already in use?

If port 9222 is occupied, Chrome will fail to start remote debugging on that port. You must either free the port or configure MediaCrawler to use an alternative port by changing `CDP_DEBUG_PORT` in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) and restarting Chrome with the new debug port (e.g., `--remote-debugging-port=9223`).

### Does CDP mode work with Chromium or Edge browsers?

Yes. Any browser supporting the Chrome DevTools Protocol—including Chromium, Microsoft Edge, and Brave—can be used with MediaCrawler's CDP mode. Ensure the browser is launched with the `--remote-debugging-port` flag and that `CDP_DEBUG_PORT` matches your configuration.