# How to Configure MediaCrawler for a Specific Platform: Complete Setup Guide

> Configure MediaCrawler for any platform by creating a custom Python config file. Follow this setup guide to launch the crawler with targeted settings for your needs.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-28

---

**You configure MediaCrawler for a specific platform by creating a Python configuration file in `config/<platform>_config.py` that overrides the global defaults in [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py), then launching the crawler with the `--platform` argument to load the targeted settings.**

MediaCrawler is a modular social media scraping framework that supports platforms like Xiaohongshu, Douyin, and Bilibili through a layered configuration system. To target a specific platform, you must understand how the tool resolves settings from base defaults to platform-specific overrides. This guide explains the exact file paths, configuration classes, and command-line arguments required to customize the crawler for any supported site.

## Understanding the Configuration Hierarchy

MediaCrawler resolves its final runtime configuration through a strict three-tier hierarchy defined across the `config/` directory. According to the source code, the system loads and merges settings in this order:

1. **Base configuration** ([`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py)) provides global defaults for headless mode, Chrome DevTools Protocol (CDP) settings, and data storage options.
2. **Platform-specific configuration** (`config/<platform>_config.py`) overrides defaults for the selected target (e.g., [`xhs_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/xhs_config.py) for Xiaohongshu, [`dy_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/dy_config.py) for Douyin).
3. **CLI arguments** ([`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py)) and environment variables apply final runtime overrides before the crawler core ([`base/base_crawler.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/base/base_crawler.py)) initializes the platform client.

The `Config` dataclass exposed in each module is imported by [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py) and instantiated based on the `--platform` argument passed at runtime.

## Step-by-Step Platform Configuration

### 1. Create the Platform Configuration File

Create a new file at `config/<platform>_config.py` following the lowercase naming convention used by existing platforms. For example, to configure Xiaohongshu support, you would edit [`config/xhs_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/xhs_config.py):

```python

# config/xhs_config.py

from dataclasses import dataclass

@dataclass
class Config:
    PLATFORM: str = "xhs"
    LOGIN_TYPE: str = "qrcode"  # Options: qrcode, phone, cookie

    HEADLESS: bool = True
    PROXY: str = ""
    FETCH_COMMENTS: bool = True
    SAVE_DATA_OPTION: str = "json"

```

### 2. Define Authentication and Connection Parameters

Set the **LOGIN_TYPE** field to specify the authentication method. Valid options are `qrcode`, `phone`, or `cookie` as implemented in the platform client modules. Additionally, configure the **PROXY** string and **HEADLESS** boolean to control browser visibility and network routing for this specific platform without affecting global defaults.

### 3. Launch with the Platform Flag

Execute the crawler by specifying the platform code that matches your configuration file prefix:

```bash
uv run main.py --platform xhs --lt qrcode --type search

```

The `--platform` argument must match the filename prefix (e.g., `xhs` for [`xhs_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/xhs_config.py)). The [`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py) module parses these arguments and merges them with the configuration dataclass before instantiating the client in `media_platform/<platform>/client.py`.

## Runtime Overrides and Storage Options

You can override any configuration value at runtime without modifying the source files. The CLI supports several key parameters that interact with the configuration system:

- `--platform`: Selects the target platform configuration file.
- `--lt` or `--login-type`: Overrides the `LOGIN_TYPE` field.
- `--type`: Specifies crawler operation mode (`search`, `detail`, or `creator`).
- `--save_data_option`: Changes the storage backend (`json`, `csv`, `excel`, or `db`).
- `--headless`: Overrides the headless browser setting.

For example, to export Xiaohongshu data to Excel using a visible browser window:

```bash
uv run main.py --platform xhs --lt qrcode --type search --save_data_option excel --headless false

```

This triggers the `ExcelStoreBase` implementation in [`store/excel_store_base.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/excel_store_base.py) to write results to `data/xhs/xhs_search_<timestamp>.xlsx`.

## Extending Configuration to New Platforms

To configure MediaCrawler for a platform not yet in the repository, implement the following file structure:

1. **Configuration**: Add `config/<new_platform>_config.py` with a `Config` dataclass defining `LOGIN_TYPE`, `HEADLESS`, and platform-specific endpoints.
2. **Data Models**: Define Pydantic schemas in `model/m_<platform>.py` (e.g., [`model/m_xhs.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/model/m_xhs.py)) to validate scraped data structures.
3. **Client Implementation**: Create `media_platform/<platform>/client.py` implementing the `login()`, `fetch_posts()`, and `fetch_comments()` methods.
4. **Registration**: The loader automatically discovers configuration files ending with [`_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/_config.py) in the `config/` directory, so no manual registry edit is required unless adding custom dispatch logic in [`media_platform/__init__.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/__init__.py).

## Summary

- MediaCrawler resolves configuration through a hierarchy: [`config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/base_config.py) → `config/<platform>_config.py` → CLI arguments parsed by [`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py).
- Create platform-specific settings in `config/<platform>_config.py` using the `Config` dataclass pattern.
- Control authentication methods via the `LOGIN_TYPE` field with options `qrcode`, `phone`, or `cookie`.
- Launch crawls using `uv run main.py --platform <code> --lt <method> --type <mode>`.
- Override storage backends at runtime with `--save_data_option` to target `ExcelStoreBase` or database configurations defined in [`config/db_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/db_config.py).

## Frequently Asked Questions

### What filename convention should I use for platform configuration files?

Name your file `config/<platform>_config.py` where `<platform>` is a lowercase code matching the `--platform` argument. For example, use [`config/dy_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/dy_config.py) for Douyin and invoke with `--platform dy`. The system automatically discovers and loads any file ending with [`_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/_config.py) in the `config/` directory.

### How do I switch between QR code and cookie authentication?

Set the `LOGIN_TYPE` field in your platform configuration file to `"qrcode"`, `"phone"`, or `"cookie"`, or override at runtime using the `--lt` argument. For example: `uv run main.py --platform xhs --lt cookie --type search`.

### Can I use a proxy for only one platform while keeping others direct?

Yes. Define the `PROXY` field in the specific platform configuration file (e.g., [`config/dy_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/dy_config.py)) rather than in [`base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/base_config.py). The system loads only the active platform's configuration at runtime, so proxy settings in other platform files remain inactive.

### Where does the configuration get merged with command-line arguments?

The merging logic resides in [`cmd_arg/arg.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/cmd_arg/arg.py), which parses CLI flags and overlays them onto the `Config` dataclass loaded from the platform-specific module. This occurs before the crawler core ([`base/base_crawler.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/base/base_crawler.py)) initializes the platform client (`media_platform/<platform>/client.py`).