# Where Are the Configuration Files Located in MediaCrawler?

> Find MediaCrawler configuration files in the main/config/ directory. Discover platform-specific modules and the shared base_config.py for all your settings.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-07-29

---

**MediaCrawler stores all runtime configuration in the `main/config/` directory, with platform-specific modules for each social network and a shared [`base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/base_config.py) for common settings.**

MediaCrawler, the open-source social media crawling framework maintained by NanmiCoder, organizes its configuration files within a dedicated Python package under the `main/` source tree. Understanding the exact location and structure of these files is essential for customizing API endpoints, authentication credentials, and database connections across supported platforms.

## Configuration Directory Structure

All configuration modules reside under the `main/config/` path in the repository. This location follows a modular architecture that separates shared defaults from platform-specific parameters, allowing each crawler to operate with isolated settings.

The directory structure follows this pattern:

```

MediaCrawler/
└── main/
    └── config/
        ├── base_config.py
        ├── zhihu_config.py
        ├── xhs_config.py
        ├── weibo_config.py
        ├── tieba_config.py
        ├── ks_config.py
        ├── dy_config.py
        ├── bilibili_config.py
        └── db_config.py

```

## Core Configuration Files Explained

### base_config.py

The [`main/config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main/config/base_config.py) module defines **shared defaults** and utility functions used across all crawling operations. This file contains common HTTP headers, retry logic parameters, and helper methods for loading environment-specific values. When adjusting global behavior like request timeouts or user-agent rotation, modify this central file to affect all platform crawlers.

### Platform-Specific Configurations

Each supported social media platform maintains its own dedicated configuration module in the same directory:

- **zhihu_config.py** – Stores Zhihu API endpoints, authentication keys, and crawling parameters
- **xhs_config.py** – Contains XiaoHongShu configuration including API tokens and pagination controls  
- **weibo_config.py** – Houses Weibo authentication cookies, API URLs, and rate-limit settings
- **tieba_config.py** – Defines Baidu Tieba forum URLs, thread limits, and scraping intervals
- **ks_config.py** – Configures Kuaishou video fetch parameters and credential handling
- **dy_config.py** – Stores Douyin-specific API keys and endpoint paths
- **bilibili_config.py** – Contains Bilibili API token placeholders and request configurations

These modules enable platform-specific customization without modifying the core `BaseCrawler` logic.

### db_config.py

The [`main/config/db_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main/config/db_config.py) file centralizes **database connection management**. It defines connection strings, cache settings, and SQLAlchemy ORM options used throughout the application. When deploying MediaCrawler with persistent storage, this file controls how the system connects to your backend database.

## How to Access Configuration Values in Code

Configuration classes are imported directly by crawler implementations at runtime. The system uses simple Python imports to retrieve settings, ensuring type-safe access to constants and methods.

### Loading Platform Configuration

To access Zhihu-specific settings in your crawler:

```python
from config.zhihu_config import ZhihuConfig

# Access a specific setting

api_endpoint = ZhihuConfig.API_URL

```

### Using Base Configuration

For common headers shared across all platforms:

```python
from config.base_config import BaseConfig

# Retrieve default headers used for all HTTP requests

default_headers = BaseConfig.DEFAULT_HEADERS

```

### Database Configuration

To initialize database connections using configured values:

```python
from config.db_config import DBConfig

# Create engine using configuration values

engine = DBConfig.create_engine()

```

## Summary

- MediaCrawler configuration files are located in the `main/config/` directory of the repository
- **base_config.py** provides shared defaults and utility functions for all crawling operations
- Each platform (Zhihu, XiaoHongShu, Weibo, etc.) has its own dedicated config module for specific endpoints and credentials
- **db_config.py** manages database connection strings and ORM settings
- Configuration values are accessed via direct Python imports from the `config` package

## Frequently Asked Questions

### Where is the main configuration directory in MediaCrawler?

The main configuration directory is located at `main/config/` in the repository root. This package contains all platform-specific modules and the shared base configuration. According to the NanmiCoder/MediaCrawler source code, core classes like `BaseCrawler` import settings from this path to retrieve runtime parameters.

### How do I add custom settings to MediaCrawler?

You can add custom settings by extending the appropriate configuration module in `main/config/`. For platform-specific values, edit the corresponding file (e.g., [`xhs_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/xhs_config.py) for XiaoHongShu). For global settings that apply to all crawlers, define new constants in [`base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/base_config.py) where utility functions for environment-specific loading are also provided.

### Which file should I edit for database connection settings?

Database configuration is handled exclusively in [`main/config/db_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main/config/db_config.py). This file contains connection strings, pool settings, and SQLAlchemy ORM configurations. Modify this file when changing your database backend or adjusting connection parameters like timeouts, SSL settings, or cache options.

### Can I use environment variables with MediaCrawler configs?

Yes, the configuration system supports environment variables through utility functions defined in [`main/config/base_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main/config/base_config.py). You can leverage these helpers to load sensitive credentials like API keys and database URLs from environment variables, keeping secrets out of version control while maintaining the structured configuration approach.