How to Run DeepSeek-R1 Models Locally Using vLLM: Complete Setup Guide

You can run DeepSeek-R1 distilled models locally using vLLM by installing the framework with pip install vllm and launching the server with vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768.

The deepseek-ai/DeepSeek-R1 repository provides a state-of-the-art reasoning model that requires specific serving infrastructure to run efficiently. This guide explains how to run DeepSeek-R1 models locally using vLLM, covering installation, configuration, and optimization for the distilled variants that are compatible with this serving framework.

Understanding DeepSeek-R1 Architecture

DeepSeek-R1 is built on the DeepSeek-V3-Base architecture and employs a Mixture-of-Experts (MoE) design. According to the repository's README.md (lines 71-75), the model contains 671 billion total parameters but activates only 37 billion parameters per token during inference, enabling efficient processing despite its massive scale.

The model supports an extended context length of 128,000 tokens and is trained via large-scale reinforcement learning without supervised fine-tuning, giving it strong chain-of-thought reasoning capabilities (README.md lines 51-55).

Installation and Prerequisites

Before you can run DeepSeek-R1 models locally using vLLM, ensure your environment meets the following requirements:

• Python ≥ 3.9
• CUDA ≥ 11.8
• Sufficient GPU memory for the distilled model variant (see hardware requirements below)

Install vLLM using pip:

pip install vllm

Running DeepSeek-R1 Distilled Models with vLLM

The full 671B DeepSeek-R1 model requires specialized infrastructure, but the distilled variants (such as the Qwen-based 32B and 14B models) run efficiently on vLLM. As noted in the repository documentation (README.md lines 68-70), vLLM is the primary recommended framework for these smaller, distilled versions.

Launching the vLLM Server

Use the vllm serve command to start the OpenAI-compatible API server. The following example (documented in README.md lines 174-178) demonstrates serving the 32B distilled model with tensor parallelism:

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
    --tensor-parallel-size 2 \
    --max-model-len 32768 \
    --enforce-eager

Key parameters explained:

• --tensor-parallel-size 2: Distributes the model across 2 GPUs using tensor parallelism
• --max-model-len 32768: Sets the maximum sequence length (32,768 tokens for this distilled variant)
• --enforce-eager: Disables CUDA graph optimizations to avoid potential compatibility issues with MoE architectures

Querying the Model

Once the server is running, interact with it via the OpenAI-compatible HTTP API at http://localhost:8000/v1/completions or http://localhost:8000/v1/chat/completions.

Optimizing Inference Parameters

To achieve optimal reasoning quality when you run DeepSeek-R1 models locally using vLLM, adhere to the configuration guidelines in the repository (README.md lines 90-92):

• Temperature: Set between 0.5 and 0.7, with 0.6 being the recommended default. Values outside this range may cause repetitive or incoherent reasoning outputs.
• System Prompts: Do not use system prompts. Embed all instructions directly within the user prompt to avoid interfering with the model's reasoning process.
• Reasoning Extraction: The model outputs reasoning content within `

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}