What Social Media Platforms Are Supported by MediaCrawler: The Complete 2024 Guide

MediaCrawler officially supports seven major Chinese social media platforms—XiaoHongShu (xhs), Douyin (dy), Kuaishou (ks), Bilibili (bili), Weibo (wb), Tieba (tieba), and Zhihu (zhihu)—each mapped to specific crawler implementations via the CrawlerFactory class.

MediaCrawler is an asynchronous Python web-scraping framework designed for extracting data from Chinese social media networks. As implemented in the NanmiCoder/MediaCrawler repository, the system architecture defines a clear registry of supported platforms that determines which crawler modules are available for data extraction tasks.

Complete List of Supported Social Media Platforms

The framework maintains a hard-coded registry of seven platforms, documented in the Project Architecture file (docs/项目架构文档.md)【/cache/repos/github.com/NanmiCoder/MediaCrawler/main/docs/项目架构文档.md#L9-L20】 and implemented in the core factory class. Each platform is identified by a short code used for CLI and programmatic access.

Platform	Code	Primary Capabilities
XiaoHongShu (小红书)	xhs	Search notes, fetch note details, retrieve creator info
Douyin (抖音)	dy	Search videos, fetch video details, retrieve creator info
Kuaishou (快手)	ks	Search videos, fetch video details, retrieve creator info
Bilibili (B站)	bili	Search videos, fetch video details, retrieve UP-owner info
Weibo (微博)	wb	Search posts, fetch post details, retrieve author info
Tieba (百度贴吧)	tieba	Search threads, fetch thread details
Zhihu (知乎)	zhihu	Search questions/answers, fetch content details, retrieve answerer info

Platform Registration Architecture

The supported platforms are registered through the CrawlerFactory class defined in main.py【/cache/repos/github.com/NanmiCoder/MediaCrawler/main/main.py#L50-L58】. This factory maps each platform code to its concrete crawler implementation, creating the abstraction layer that handles platform-specific authentication and data extraction logic.

According to the architecture diagram in the documentation, each platform operates as a distinct node under the "Core Crawler Layer"【/cache/repos/github.com/NanmiCoder/MediaCrawler/main/docs/项目架构文档.md#L44-L56】. The factory instantiation pattern ensures that calling CrawlerFactory.create_crawler('xhs') returns an initialized XiaoHongShuCrawler instance, while 'dy' returns a DouYinCrawler, and so forth for all seven supported networks.

Platform Implementation Files

Each supported platform has a dedicated implementation module under the media_platform/ directory:

• media_platform/xhs/core.py – XiaoHongShu crawler implementation
• media_platform/douyin/core.py – Douyin crawler implementation
• media_platform/kuaishou/core.py – Kuaishou crawler implementation
• media_platform/bilibili/core.py – Bilibili crawler implementation
• media_platform/weibo/core.py – Weibo crawler implementation
• media_platform/tieba/core.py – Tieba crawler implementation
• media_platform/zhihu/core.py – Zhihu crawler implementation

These files constitute the concrete implementations that handle platform-specific API endpoints, rate limiting, and data parsing logic.

Usage Examples

Command Line Interface

The command-line arguments are parsed by cmd_arg/arg.py, where the --platform flag expects one of the seven official codes. To run a crawl operation:

# Crawl Douyin for videos matching "travel"

python -m main --platform dy --type search --keyword travel

# Fetch XiaoHongShu notes about "skincare"

python -m main --platform xhs --type search --keyword skincare

Programmatic API

You can instantiate specific platform crawlers directly through the factory:

from main import CrawlerFactory

# Initialize XiaoHongShu crawler

crawler = CrawlerFactory.create_crawler('xhs')
await crawler.start()

# Initialize Bilibili crawler  

bili_crawler = CrawlerFactory.create_crawler('bili')
await bili_crawler.start()

Extending Platform Support

While MediaCrawler currently supports seven platforms, the factory architecture allows for extension. To add a new platform, you would implement a new *Crawler class in media_platform/<new_platform>/core.py and register it in the CRAWLERS map in main.py:

# In main.py

CrawlerFactory.CRAWLERS["new_platform"] = NewPlatformCrawler

This registration pattern maintains the abstraction layer that the rest of the framework depends on for configuration management and logging.

Summary

• MediaCrawler supports seven Chinese social media platforms: XiaoHongShu (xhs), Douyin (dy), Kuaishou (ks), Bilibili (bili), Weibo (wb), Tieba (tieba), and Zhihu (zhihu).
• Platform selection occurs through the CrawlerFactory class in main.py, which maps short codes to concrete crawler implementations.
• Each platform has dedicated implementation files under media_platform/<platform_code>/core.py handling specific data extraction logic.
• Command-line usage requires the --platform flag with one of the supported codes, as defined in cmd_arg/arg.py.
• The architecture is extensible via the factory pattern, allowing new platforms to be registered by updating the CRAWLERS dictionary.

Frequently Asked Questions

How many social media platforms does MediaCrawler support?

MediaCrawler supports seven social media platforms as of the latest implementation. These are all Chinese platforms: XiaoHongShu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu. The complete list is documented in the architecture documentation at docs/项目架构文档.md and hard-coded in the CrawlerFactory class.

What is the platform code for XiaoHongShu in MediaCrawler?

The platform code for XiaoHongShu (小红书) is xhs. This short code is used when initializing the crawler via CrawlerFactory.create_crawler('xhs') or when using the command-line interface with --platform xhs. The code maps to the XiaoHongShuCrawler class in media_platform/xhs/core.py.

Can I add custom platforms to MediaCrawler?

Yes, you can extend MediaCrawler to support additional platforms by implementing the crawler logic in a new module under media_platform/ and registering the platform in the CRAWLERS dictionary in main.py. The factory pattern requires adding a mapping between a unique platform code string and your new crawler class, following the existing implementation pattern used for the seven supported platforms.

Does MediaCrawler support international platforms like Instagram or Twitter?

No, the current implementation only supports Chinese social media platforms. The CrawlerFactory in main.py explicitly maps only the seven Chinese platform codes (xhs, dy, ks, bili, wb, tieba, zhihu) to their respective crawler classes. There are no existing implementations for Instagram, Twitter, Facebook, or other international platforms in the repository structure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}