How to Run MediaCrawler API Server: Complete Setup and Configuration Guide

You can start the MediaCrawler API server by installing dependencies from requirements.txt, configuring your .env file, and launching uvicorn api.main:app --host 0.0.0.0 --port 8080.

The MediaCrawler API server is a FastAPI application that provides a programmable REST and WebSocket interface for controlling media platform crawlers. Located in the NanmiCoder/MediaCrawler repository, this server exposes endpoints to initiate crawls, download data, and monitor real-time progress without directly invoking the command-line interface.

Prerequisites and Installation

Before running the server, you must install the Python environment and project dependencies.

Install Python Dependencies

The repository uses uv for fast dependency management, but standard pip works as well. Install all required packages including FastAPI, Uvicorn, Playwright, and database drivers:

# Install uv (recommended)

pip install uv

# Install dependencies

uv pip install -r requirements.txt

The [requirements.txt](https://github.com/NanmiCoder/MediaCrawler/blob/main/requirements.txt) file pins exact versions for fastapi, uvicorn, httpx, playwright, and other utilities like wordcloud required by the crawlers.

Configuration Setup

The API server relies on environment variables for runtime behavior. All configuration options are defined in the example file and loaded via python-dotenv.

Environment Variables

Copy the example configuration file and customize values for your use case:

cp .env.example .env

Edit the .env file to set options such as SAVE_DATA_OPTION (json, csv, db), ENABLE_GET_WORDCLOUD, and platform-specific settings. No secret keys are hardcoded in the API itself; all sensitive data resides in this configuration file. See the full template in .env.example.

Optional WebUI Build

If you want to use the graphical web interface, compile the front-end assets first:

cd api/webui
npm install
npm run build

This creates the dist/ directory containing static assets. If you skip this step, the server will still function—the root endpoint at / will return a JSON message indicating the UI is missing rather than serving index.html. The static mounting logic is implemented in [api/main.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/main.py) lines 42-76.

Launching the MediaCrawler API Server

The FastAPI application instance is defined in [api/main.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/main.py) and can be started using two equivalent methods.

Using Uvicorn (Recommended)

For development with hot-reload support, use Uvicorn directly:

uvicorn api.main:app --host 0.0.0.0 --port 8080 --reload

This command binds to all interfaces on port 8080 and automatically restarts the server when code changes are detected.

Using Python Module

Alternatively, run the module directly to start the server:

python -m api.main

Verification Endpoints

Once running, verify the server health and configuration:

# Check server status

curl http://localhost:8080/api/health

# → {"status":"ok"}

# List supported platforms

curl http://localhost:8080/api/config/platforms

# → JSON array with platform entries (xhs, dy, ks, etc.)

The health check endpoint is defined at line 79 in [api/main.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/main.py), while the environment verification endpoint at /api/env/check runs uv run main.py --help to confirm the CLI environment is properly configured.

Architecture Overview

The API server is composed of several modular components:

FastAPI Application Core – The main app factory in [api/main.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/main.py) (lines 36-63) configures CORS, mounts static files, and registers exception handlers.

Router Modules – Business logic is separated into three primary routers:

• [api/routers/crawler.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/routers/crawler.py) – Handles /api/crawler/* endpoints for starting, stopping, and monitoring crawler jobs.
• [api/routers/data.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/routers/data.py) – Provides data export endpoints for JSON, CSV, Excel, and database formats.
• [api/routers/websocket.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/routers/websocket.py) – Manages real-time WebSocket connections for streaming crawl progress updates.

Request Validation – All API payloads are validated using Pydantic models defined in [api/schemas/crawler.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/schemas/crawler.py).

Cleanup Handlers – The server registers shutdown handlers that gracefully terminate any CDP (Chromium DevTools Protocol) browsers launched by crawlers when the process receives SIGINT (Ctrl+C).

Summary

• Install dependencies using uv pip install -r requirements.txt or standard pip.
• Configure your environment by copying .env.example to .env and adjusting settings.
• Build the optional WebUI with npm run build in the api/webui directory if you need the graphical interface.
• Launch using uvicorn api.main:app --host 0.0.0.0 --port 8080 or python -m api.main.
• Verify deployment by checking the /api/health endpoint for {"status":"ok"}.

Frequently Asked Questions

What is the default port for the MediaCrawler API server?

The documentation and examples use port 8080 by default. You can specify a different port using the --port flag when launching Uvicorn, for example: --port 3000.

Do I need to build the WebUI before running the API server?

No. The WebUI is optional. If the compiled assets are missing from api/webui/dist, the server continues to operate normally, serving JSON responses at the root endpoint instead of the web interface. The REST API and WebSocket endpoints remain fully functional without the front-end build.

How do I verify the MediaCrawler API server is running correctly?

Send a GET request to /api/health which returns {"status":"ok"}. For a deeper environment check, use /api/env/check, which executes the underlying CLI command uv run main.py --help to verify that Playwright and other core dependencies are accessible.

Where is the FastAPI application instance defined?

The main FastAPI application is instantiated in [api/main.py](https://github.com/NanmiCoder/MediaCrawler/blob/main/api/main.py). This file imports and includes routers from api/routers/crawler.py, api/routers/data.py, and api/routers/websocket.py, and configures the static file serving for the WebUI at lines 42-76.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}