# ENABLE_CDP_MODE vs Playwright Headless Mode in MediaCrawler: Key Differences Explained

> Understand ENABLE_CDP_MODE vs Playwright headless mode in MediaCrawler. Learn how MediaCrawler connects to Chrome or uses Playwright's fresh instances for optimal control.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: deep-dive
- Published: 2026-07-02

---

**ENABLE_CDP_MODE determines whether MediaCrawler connects to an existing Chrome browser via the Chrome DevTools Protocol (CDP) or launches a fresh Playwright-managed instance, while the headless flags only control whether the browser window is visible during execution.**

MediaCrawler is an open-source social media scraping framework that provides flexible browser automation strategies to balance stealth and performance. Understanding the distinction between connection mode and visibility settings is essential for configuring effective anti-detection measures and managing system resources efficiently.

## What ENABLE_CDP_MODE Controls

`ENABLE_CDP_MODE` is a boolean configuration option defined in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) (line 59) that fundamentally changes how the crawler initiates browser sessions.

When set to `True`, MediaCrawler creates a `CDPBrowserManager` instance from [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) and invokes its `launch_and_connect` method to attach to a real Chrome or Edge browser via the Chrome DevTools Protocol. This approach preserves the user's existing browser profile, including cookies, extensions, and login states, significantly improving anti-detection capabilities by making automated traffic appear indistinguishable from regular browsing.

When set to `False`, the crawler follows the standard Playwright path, launching a fresh, temporary Chromium instance that is isolated from the user's normal browsing environment.

## What Headless Mode Controls

The headless flags (`HEADLESS` and `CDP_HEADLESS`) exclusively manage UI visibility and do not affect connection strategy.

- **`HEADLESS`** (defined in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py), line 50): Applied when `ENABLE_CDP_MODE` is `False`, passed directly to `chromium.launch(headless=config.HEADLESS, ...)`
- **`CDP_HEADLESS`** (defined in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py), line 73): Applied when `ENABLE_CDP_MODE` is `True`, forwarded to `CDPBrowserManager.launch_and_connect(..., headless=config.CDP_HEADLESS)`

These flags determine whether the browser renders a visible window. Running headless (`True`) consumes fewer resources but increases detection risk, as many sites flag headless browsers as suspicious automation.

## Implementation in Platform Core Modules

Each platform implementation branches on `ENABLE_CDP_MODE` to select the appropriate launch strategy. In [`media_platform/zhihu/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/zhihu/core.py) (lines 58-90), the logic appears as:

```python
if config.ENABLE_CDP_MODE:
    # CDP path - connects to existing Chrome

    browser_context = await self.launch_browser_with_cdp(
        playwright, playwright_proxy, self.user_agent,
        headless=config.CDP_HEADLESS,
    )
else:
    # Standard Playwright launch - creates new Chromium

    browser_context = await self.launch_browser(
        chromium, playwright_proxy, self.user_agent,
        headless=config.HEADLESS,
    )

```

The `launch_browser_with_cdp` method initializes the CDP connection, while `launch_browser` handles standard Playwright instantiation. Similar branching logic exists across all platform cores, including [`media_platform/xhs/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/xhs/core.py) and others.

## Configuration Patterns and Use Cases

### Maximum Stealth (Recommended for Production)

```python

# config/base_config.py

ENABLE_CDP_MODE = True      # Connect to real Chrome

CDP_HEADLESS = False        # Show the browser window

CDP_CONNECT_EXISTING = True # Reuse existing Chrome process

```

This configuration leverages the user's genuine browser profile and visible UI, minimizing detection probability. The crawler connects to an already-running Chrome instance rather than spawning a new process.

### Lightweight Testing (Development Environment)

```python

# config/base_config.py

ENABLE_CDP_MODE = False     # Use Playwright's Chromium

HEADLESS = True             # Run without visible window

AUTO_CLOSE_BROWSER = False  # Keep browser open for debugging

```

This pattern launches a fresh, headless browser suitable for rapid development iterations where anti-detection is not critical.

### Headless CDP (Server Deployment)

```python

# config/base_config.py

ENABLE_CDP_MODE = True
CDP_HEADLESS = True

```

Even when connecting via CDP, you can run Chrome in headless mode by starting the remote browser with the `--headless` flag. The `CDPBrowserManager` passes this configuration to `playwright.chromium.connect_over_cdp` as implemented in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) (lines 13-31).

## Anti-Detection and Resource Implications

**Profile Persistence**: CDP mode maintains persistent sessions across crawler runs because it operates within the user's actual browser profile. Standard Playwright mode creates temporary profiles that are deleted when `AUTO_CLOSE_BROWSER` triggers (unless explicitly disabled).

**Process Management**: When `ENABLE_CDP_MODE` is `True` and `CDP_CONNECT_EXISTING` is enabled, MediaCrawler attaches to an existing browser process rather than spawning new ones. This reduces memory overhead and avoids the process churn associated with launching fresh Chromium instances.

**Detection Risk**: Headless detection remains a concern regardless of connection mode. Even with CDP enabled, setting `CDP_HEADLESS=True` may trigger anti-bot measures on sophisticated platforms. The command-line argument parser in [`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py) exposes these flags independently, allowing runtime overrides without modifying configuration files.

## Summary

- **ENABLE_CDP_MODE** switches between connecting to an existing Chrome via CDP (`True`) and launching a fresh Playwright Chromium instance (`False`)
- **Headless flags** only control UI visibility and are independent of connection mode
- CDP mode preserves user profiles, cookies, and extensions for superior anti-detection
- Standard Playwright mode creates isolated, temporary browser sessions suitable for contained scraping
- Configuration options are defined in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) and applied in platform-specific core modules under `media_platform/*/`

## Frequently Asked Questions

### Can I use headless mode when ENABLE_CDP_MODE is enabled?

Yes. Set `ENABLE_CDP_MODE = True` and `CDP_HEADLESS = True` in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py). The crawler will connect to your Chrome instance via CDP, but the remote Chrome must be started with headless flags (e.g., `--headless=new`). The `CDP_HEADLESS` parameter is passed to `connect_over_cdp` in [`tools/cdp_browser.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/tools/cdp_browser.py) to ensure consistency.

### Which configuration is best for avoiding bot detection?

Use `ENABLE_CDP_MODE = True` with `CDP_HEADLESS = False` and `CDP_CONNECT_EXISTING = True`. This connects to a visible Chrome instance with your real user profile, cookies, and extensions intact. Headless browsers are easily flagged by sophisticated anti-bot systems, while CDP-connected real browsers appear as legitimate user traffic.

### Where do I configure these settings in MediaCrawler?

All browser configuration flags reside in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py). Lines 50-73 contain the relevant settings: `HEADLESS` (line 50) for standard mode, `ENABLE_CDP_MODE` (line 59) for CDP toggle, and `CDP_HEADLESS` (line 73) for CDP visibility. You can also override these via command-line arguments defined in [`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py).

### What happens if I set HEADLESS=True but ENABLE_CDP_MODE=True?

The `HEADLESS` variable is ignored when `ENABLE_CDP_MODE` is active. In this state, only `CDP_HEADLESS` affects visibility. The platform core modules explicitly check `config.ENABLE_CDP_MODE` first and use the corresponding headless flag (`CDP_HEADLESS` vs `HEADLESS`) for that specific launch path.