How to Configure a Custom Browser Path for CDP Mode in MediaCrawler

To configure a custom browser path for CDP Mode in MediaCrawler, set the CUSTOM_BROWSER_PATH variable in config/base_config.py to the absolute path of your Chrome or Edge executable.

MediaCrawler leverages the Chrome DevTools Protocol (CDP) to control real browser instances for web scraping tasks. When CDP mode is enabled, the crawler checks for a custom browser path before falling back to automatic detection, allowing you to specify exact browser binaries or portable builds according to the NanmiCoder/MediaCrawler source code.

How CDP Browser Path Resolution Works

When ENABLE_CDP_MODE is set to True in your configuration, MediaCrawler delegates browser launching to the CDPBrowserManager class. The path resolution logic resides in tools/cdp_browser.py within the _get_browser_path method (lines 97–106).

The code checks for your custom path first:

if config.CUSTOM_BROWSER_PATH and os.path.isfile(config.CUSTOM_BROWSER_PATH):
    # use the custom path

    return config.CUSTOM_BROWSER_PATH

If CUSTOM_BROWSER_PATH is empty or points to a non-existent file, MediaCrawler automatically invokes BrowserLauncher.detect_browser_paths from tools/browser_launcher.py to find system-installed Chrome or Edge binaries.

Step-by-Step Configuration

Locate the Configuration File

Open config/base_config.py in your MediaCrawler installation. This file contains all global configuration constants, including CDP-related settings around line 69.

Set the Custom Browser Path

Assign the absolute path to your browser executable to the CUSTOM_BROWSER_PATH constant. This path must point to the actual binary file, not a directory or symbolic link.

Windows example:

# config/base_config.py

CUSTOM_BROWSER_PATH = r"C:\Program Files\Google\Chrome\Application\chrome.exe"

macOS example:

# config/base_config.py

CUSTOM_BROWSER_PATH = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

Linux/Portable Chromium example:

# config/base_config.py

CUSTOM_BROWSER_PATH = "/opt/chromium/chrome"

Verify Path Validity

The _get_browser_path method validates your configuration using os.path.isfile(). Ensure the file exists and has execute permissions. An incorrect path triggers silent fallback to auto-detection, which may cause version mismatches if multiple browsers are installed.

Verification Through Code

To confirm your custom path is active before launching a crawl, inspect the configuration at runtime:

import config
from tools.cdp_browser import CDPBrowserManager
from playwright.async_api import async_playwright

async def verify_browser_path():
    if not config.CUSTOM_BROWSER_PATH:
        print("No custom browser path configured")
    else:
        print(f"Using custom browser: {config.CUSTOM_BROWSER_PATH}")

    async with async_playwright() as p:
        manager = CDPBrowserManager()
        await manager.launch_and_connect(p)
        # CDP manager will use the specified executable

# asyncio.run(verify_browser_path())

Summary

• Primary configuration occurs in config/base_config.py via the CUSTOM_BROWSER_PATH constant.
• Validation logic in tools/cdp_browser.py (lines 97–106) verifies the path exists before use.
• Fallback behavior automatically detects Chrome/Edge via BrowserLauncher.detect_browser_paths when no custom path is set.
• Path requirements must specify the executable file directly, not containing directories.
• Cross-platform support works for Windows, macOS, and Linux binaries including portable Chromium builds.

Frequently Asked Questions

What happens if my custom browser path is ignored?

If MediaCrawler ignores your custom path, verify that CUSTOM_BROWSER_PATH points to an executable file and not a directory. The code explicitly checks os.path.isfile(config.CUSTOM_BROWSER_PATH) in tools/cdp_browser.py. If the file does not exist or lacks read permissions, the system silently falls back to auto-detection.

Can I use Chromium or Edge instead of Chrome?

Yes. The CUSTOM_BROWSER_PATH constant accepts any Chromium-based browser executable, including Microsoft Edge, Brave, or custom Chromium builds. Ensure the binary supports the Chrome DevTools Protocol and matches the Playwright version requirements specified in MediaCrawler's dependencies.

Does custom browser path configuration work with headless mode?

Yes. Once you configure a custom browser path for CDP Mode in MediaCrawler, headless settings are controlled separately through HEADLESS settings in config/base_config.py. The custom path specifies which binary launches, while headless mode determines whether the browser window displays.

Where does MediaCrawler search for browsers when no custom path is set?

When CUSTOM_BROWSER_PATH is empty or invalid, MediaCrawler invokes detect_browser_paths from tools/browser_launcher.py. This method scans standard installation directories for Chrome and Edge executables based on your operating system, selecting the first valid match found.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}