# How to Use MediaCrawler for Kuaishou Data: Configuration, Modes, and Implementation

> Learn how to use MediaCrawler for Kuaishou data. Explore configuration, crawling modes, and async Playwright & GraphQL implementation for video metadata, comments, and creator profiles.

- Repository: [程序员阿江-Relakkes/MediaCrawler](https://github.com/NanmiCoder/MediaCrawler)
- Tags: how-to-guide
- Published: 2026-06-26

---

**MediaCrawler enables automated collection of Kuaishou video metadata, comments, and creator profiles through three configurable crawling modes backed by an async Playwright and GraphQL architecture.**

MediaCrawler is an open-source multi-platform crawler framework maintained by NanmiCoder that supports automated data extraction from Chinese social media platforms. This guide explains how to use MediaCrawler for Kuaishou data collection, covering the specific configuration files, core components, and execution flow required to scrape video information and user-generated content.

## MediaCrawler Architecture for Kuaishou

The Kuaishou implementation follows a modular design pattern with clear separation between configuration, client operations, and data persistence. Understanding these components is essential for customizing the crawler behavior.

### Configuration Layer

The [`config/ks_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/ks_config.py) file defines runtime parameters including `CRAWLER_TYPE`, concurrency limits, and target ID lists. This configuration drives the behavior of the `KuaishouCrawler` class in [`media_platform/kuaishou/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/core.py). Key parameters include `MAX_CONCURRENCY_NUM` for controlling async workers and `CRAWLER_MAX_SLEEP_SEC` for rate limiting.

### URL Parsing and ID Extraction

The [`media_platform/kuaishou/help.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/help.py) module provides utility functions to extract video and creator identifiers from Kuaishou URLs. The `parse_video_info_from_url` function returns a `VideoUrlInfo` Pydantic model defined in [`model/m_kuaishou.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/model/m_kuaishou.py), which standardizes the extraction of video IDs from various URL formats.

### Core Crawler and API Client

The `KuaishouCrawler` class in [`media_platform/kuaishou/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/core.py) orchestrates browser initialization via `launch_browser` or `launch_browser_with_cdp`, session management, and task scheduling. It utilizes `KuaiShouClient` from [`media_platform/kuaishou/client.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/client.py) to execute GraphQL queries against Kuaishou's private endpoints for search, detail, and creator data.

### Authentication Handling

Session management is handled by `KuaishouLogin` in [`media_platform/kuaishou/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/login.py), supporting both QR-code and cookie-based authentication methods. The crawler validates session status using `ks_client.pong` before executing data collection tasks.

### Storage Implementations

Data persistence is abstracted through [`store/kuaishou/_store_impl.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/kuaishou/_store_impl.py), which implements handlers for CSV, JSON/JSONL, SQLite, and MongoDB formats via `AsyncFileWriter` and database-specific adapters. The storage layer exports `update_kuaishou_video`, `batch_update_ks_video_comments`, and `save_creator` functions.

## Supported Crawling Modes

MediaCrawler supports three distinct operational modes for Kuaishou data collection, controlled by the `CRAWLER_TYPE` configuration parameter in [`config/ks_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/ks_config.py).

- **Search Mode**: Discovers videos by keyword using `search_info_by_keyword` in the client. Ideal for trend analysis and content discovery.
- **Detail Mode**: Retrieves specific video metadata and comments using `get_video_info` and `get_video_all_comments`. Requires video IDs or URLs in `KS_SPECIFIED_ID_LIST`.
- **Creator Mode**: Scrapes complete creator profiles and their video catalogs using `get_all_videos_by_creator`. Extracts all videos from specific user accounts.

## Step-by-Step Configuration

To initiate Kuaishou data collection, configure the environment and target parameters before execution.

First, install dependencies:

```bash
pip install -r requirements.txt

```

Create a `.env` file based on `.env.example`. For QR-code login, leave `COOKIES` empty; the crawler will display a QR image in the console during execution via `KuaishouLogin`.

Configure [`config/ks_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/ks_config.py) for detail mode:

```python

# config/ks_config.py

CRAWLER_TYPE = "detail"  # Options: "search", "detail", "creator"

KS_SPECIFIED_ID_LIST = [
    "https://www.kuaishou.com/short-video/3x3zxz4mjrsc8ke",
    "3xf8enb8dbj6uig",  # Raw video ID format also supported

]
MAX_CONCURRENCY_NUM = 5
CRAWLER_MAX_SLEEP_SEC = 2  # Rate limiting between requests

CRAWLER_MAX_NOTES_COUNT = 20  # Maximum items to fetch

```

## Implementing the Crawl Workflow

Once configured, execute the crawler via [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py) or programmatically using the core classes.

Command-line execution:

```bash
python main.py

```

The platform selector in [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py) automatically instantiates `KuaishouCrawler` when `config.PLATFORM` is set appropriately.

Parsing Kuaishou URLs programmatically:

```python
from media_platform.kuaishou.help import parse_video_info_from_url

url = "https://www.kuaishou.com/short-video/3x3zxz4mjrsc8ke"
info = parse_video_info_from_url(url)

# Returns VideoUrlInfo(video_id='3x3zxz4mjrsc8ke', url_type='normal')

print(info.video_id)  # Output: 3x3zxz4mjrsc8ke

```

Direct programmatic invocation:

```python
import asyncio
from media_platform.kuaishou.core import KuaishouCrawler

async def run_crawler():
    crawler = KuaishouCrawler()
    await crawler.start()  # Executes based on config.CRAWLER_TYPE

asyncio.run(run_crawler())

```

The `KuaishouCrawler.start()` method in [`media_platform/kuaishou/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/core.py) handles the complete lifecycle: browser launch (via `launch_browser` or `launch_browser_with_cdp` depending on `ENABLE_CDP_MODE`), session validation through `ks_client.pong`, login via `KuaishouLogin` if needed, and concurrent task execution limited by `MAX_CONCURRENCY_NUM`.

## Data Storage Options

The storage implementation in [`store/kuaishou/_store_impl.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/kuaishou/_store_impl.py) supports multiple backends. By default, data writes to CSV using `AsyncFileWriter`.

To switch storage backends, modify the import in [`store/kuaishou/__init__.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/kuaishou/__init__.py):

- **CSV/JSON**: Default `AsyncFileWriter` implementation
- **MongoDB**: Use `KuaishouMongoStoreImplement`
- **SQLite**: SQLAlchemy-based implementation

Key storage functions include:

- `kuaishou_store.update_kuaishou_video()` for video metadata
- `kuaishou_store.batch_update_ks_video_comments()` for comment threads
- `kuaishou_store.save_creator()` for creator profiles

## Summary

- MediaCrawler provides three Kuaishou crawling modes: **search**, **detail**, and **creator**, configured via `CRAWLER_TYPE` in [`config/ks_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/ks_config.py).
- The architecture separates concerns between [`media_platform/kuaishou/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/core.py) (orchestration), [`media_platform/kuaishou/client.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/client.py) (API calls), and [`store/kuaishou/_store_impl.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/kuaishou/_store_impl.py) (persistence).
- Authentication supports both QR-code and cookie-based methods through [`media_platform/kuaishou/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/login.py).
- URL parsing utilities in [`media_platform/kuaishou/help.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/help.py) handle extraction of video and creator IDs from standard Kuaishou URLs.
- Execution entry point is [`main.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/main.py), with configurable concurrency and rate limiting via `MAX_CONCURRENCY_NUM` and `CRAWLER_MAX_SLEEP_SEC`.

## Frequently Asked Questions

### How do I extract video IDs from Kuaishou URLs?

Use the `parse_video_info_from_url` function from [`media_platform/kuaishou/help.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/help.py). This function accepts a standard Kuaishou URL like `https://www.kuaishou.com/short-video/3x3zxz4mjrsc8ke` and returns a `VideoUrlInfo` Pydantic model containing the extracted `video_id` and `url_type` fields defined in [`model/m_kuaishou.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/model/m_kuaishou.py).

### What is the difference between CDP mode and standard Playwright mode?

CDP (Chrome DevTools Protocol) mode, enabled by setting `ENABLE_CDP_MODE = True` in [`config/ks_config.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/config/ks_config.py), launches the browser using `launch_browser_with_cdp` in [`media_platform/kuaishou/core.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/core.py) for advanced debugging and proxy capabilities. Standard mode uses Playwright's `launch_browser` method without CDP hooks.

### How do I change the data storage format from CSV to MongoDB?

Modify the storage implementation in [`store/kuaishou/__init__.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/store/kuaishou/__init__.py) to import `KuaishouMongoStoreImplement` instead of the default CSV writer. The storage layer exports standardized functions like `update_kuaishou_video` and `batch_update_ks_video_comments` that work regardless of the backend selected.

### Why does the crawler require login, and what methods are supported?

Kuaishou requires authentication for accessing video details and creator data. The `KuaishouLogin` class in [`media_platform/kuaishou/login.py`](https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/kuaishou/login.py) supports QR-code authentication (displayed in console) and cookie-based authentication (set via the `COOKIES` environment variable). The crawler validates the session using `ks_client.pong` before executing tasks.