How to Use QR Code Login in MediaCrawler: Complete Implementation Guide

MediaCrawler supports QR code authentication via the --lt qrcode CLI flag, which launches a Playwright browser instance to render the platform's login QR code, pauses execution until you scan it with the official mobile app, and automatically persists the resulting cookies to Redis or local file cache for subsequent scraping sessions.

MediaCrawler is an open-source social media scraping framework that supports three authentication methods: QR code, phone SMS, and cookie-based login. The QR code approach is the most convenient for long-running data collection tasks because it leverages your existing mobile app authentication without requiring SMS verification codes or manual cookie extraction.

How QR Code Login Works in MediaCrawler

The QR code login implementation follows a stateful browser automation pattern that bridges mobile app authentication with headless web scraping.

The Authentication Flow

According to the architecture documentation in docs/项目架构文档.md (lines 502-506), the QR login follows this sequence: 显示二维码 → 等待扫描 → 扫描成功? → 保存Cookie (Display QR → Wait for Scan → Scan Successful? → Save Cookie).

When you initiate a crawl with QR login enabled, MediaCrawler performs the following steps:

1. Launch Playwright Context: The crawler initializes a browser context via main.py and navigates to the target platform's login page.
2. Render QR Code: The framework captures the QR code image from the login page and displays it in your terminal or browser window.
3. Poll for Authentication: The script continuously checks the page state (implemented in tools/crawler_util.py with UX enhancements at line 95) until the mobile app confirms the scan.
4. Extract and Store Cookies: Once authenticated, Playwright extracts the session cookies and stores them via the CacheFactory implementation (see cache/cache_factory.py and cache/redis_cache.py).
5. Reuse Session: Subsequent requests automatically use the cached cookies, eliminating the need to scan again until the session expires.

Command-Line Configuration

The login type is defined as an enum in cmd_arg/arg.py (lines 52-58), where LoginTypeEnum includes the QRCODE option. This value is consumed by the CrawlerStartRequest schema in api/schemas/crawler.py (lines 38-66), which carries the login_type parameter through to the crawler core.

Implementation Details

Understanding the source code architecture helps you troubleshoot authentication issues and customize the login behavior.

Enum Definition in cmd_arg/arg.py

The --lt (login type) CLI argument is registered in the command-line parser:

# In cmd_arg/arg.py (lines 52-58)

class LoginTypeEnum(str, Enum):
    QRCODE = "qrcode"
    SMS = "sms"
    COOKIE = "cookie"

When you pass --lt qrcode, the parser validates the input against this enum and passes it to the crawler initialization.

Schema Validation in api/schemas/crawler.py

The CrawlerStartRequest Pydantic model ensures type safety for the login configuration:

# In api/schemas/crawler.py (lines 38-66)

class CrawlerStartRequest(BaseModel):
    platform: str
    login_type: LoginTypeEnum = LoginTypeEnum.QRCODE
    # ... additional fields

This schema defaults to QRCODE if no login type is specified, making QR login the implicit standard for most crawling operations.

Browser Automation in tools/crawler_util.py

The actual QR code display and waiting logic resides in tools/crawler_util.py. At line 95, the code includes specific UX handling for the QR code border to ensure the terminal-rendered image scans correctly with mobile devices.

Step-by-Step Usage Guide

You can trigger QR code login through either the command-line interface or the WebUI.

CLI Method

Pass the --lt qrcode flag (or -lt qrcode) to any main.py invocation:

# Search mode with QR login (Xiaohongshu example)

uv run main.py --platform xhs --lt qrcode --type search

# Detail mode with QR login (Douyin example)

uv run main.py --platform dy --lt qrcode --type detail

After execution, a browser window or terminal image will display the platform's QR code. Scan it with the official mobile app (e.g., Xiaohongshu or Douyin) and confirm the login. The script will automatically detect the successful authentication and proceed with the crawl.

WebUI Method

For a graphical interface, start the API server:

uv run uvicorn api.main:app --port 8080 --reload

Open http://localhost:8080 in your browser, select "QR Code Login" from the login type dropdown, and scan the rendered QR code. The WebUI uses the same CrawlerStartRequest schema to initialize the authenticated session.

Cookie Persistence and Cache Management

Once the QR scan completes successfully, MediaCrawler extracts the Cookie headers from the Playwright browser context. The framework supports two cache backends:

• Redis: Configured via cache/redis_cache.py for distributed crawling scenarios
• Local File: Default file-based storage in cache/cache_factory.py

The cached session persists across crawler restarts. You only need to re-scan the QR code when the platform's cookies expire or if you explicitly clear the cache.

Summary

• QR code login in MediaCrawler is enabled via the --lt qrcode CLI flag defined in cmd_arg/arg.py.
• The implementation uses Playwright to render the platform login page and extract cookies after mobile app authentication.
• The LoginTypeEnum.QRCODE value is propagated through the CrawlerStartRequest schema in api/schemas/crawler.py.
• Post-authentication cookies are stored via the CacheFactory architecture, supporting both Redis and local file backends.
• The entire flow is documented in the architecture diagram at docs/项目架构文档.md (lines 502-506), visualizing the QR generation, scan polling, and cookie persistence stages.

Frequently Asked Questions

What platforms support QR code login in MediaCrawler?

MediaCrawler supports QR code authentication for all major Chinese social media platforms implemented in the framework, including Xiaohongshu (xhs), Douyin (dy), and Bilibili. The QR login method works with any platform that provides a web-based QR code login endpoint accessible via Playwright.

Where are cookies stored after QR code authentication?

After successful QR authentication, cookies are stored according to your cache configuration in cache/cache_factory.py. By default, MediaCrawler uses a local file cache, but you can configure Redis persistence by modifying the cache backend settings. The storage path is typically in the project's cache directory or your configured Redis instance.

Can I use QR code login without the CLI?

Yes, you can use the WebUI method by running uv run uvicorn api.main:app --port 8080 --reload and accessing http://localhost:8080. The WebUI provides a dropdown to select "QR Code Login" and will display the QR code in the browser window, allowing you to authenticate without using terminal commands.

How long does the authenticated session last?

Session duration depends on the specific platform's cookie expiration policy. MediaCrawler reuses cached cookies from the CacheFactory until they expire or are invalidated by the platform. If you encounter authentication errors during a crawl, simply re-run the command with --lt qrcode to generate a fresh QR code and renew the session.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}