# How to Run DeepSeek-R1 Models Locally Using vLLM: Complete Setup Guide

> Easily run DeepSeek-R1 models locally with vLLM. Follow this guide to install vLLM and launch the DeepSeek-R1 server for efficient local inference. Get started now.

- Repository: [DeepSeek/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1)
- Tags: how-to-guide
- Published: 2026-02-27

---

**You can run DeepSeek-R1 distilled models locally using vLLM by installing the framework with `pip install vllm` and launching the server with `vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768`.**

The `deepseek-ai/DeepSeek-R1` repository provides a state-of-the-art reasoning model that requires specific serving infrastructure to run efficiently. This guide explains how to run DeepSeek-R1 models locally using vLLM, covering installation, configuration, and optimization for the distilled variants that are compatible with this serving framework.

## Understanding DeepSeek-R1 Architecture

DeepSeek-R1 is built on the DeepSeek-V3-Base architecture and employs a **Mixture-of-Experts (MoE)** design. According to the repository's [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) (lines 71-75), the model contains **671 billion total parameters** but activates only **37 billion parameters** per token during inference, enabling efficient processing despite its massive scale.

The model supports an **extended context length of 128,000 tokens** and is trained via large-scale reinforcement learning without supervised fine-tuning, giving it strong chain-of-thought reasoning capabilities (README.md lines 51-55).

## Installation and Prerequisites

Before you can run DeepSeek-R1 models locally using vLLM, ensure your environment meets the following requirements:

- **Python** ≥ 3.9
- **CUDA** ≥ 11.8
- Sufficient GPU memory for the distilled model variant (see hardware requirements below)

Install vLLM using pip:

```bash
pip install vllm

```

## Running DeepSeek-R1 Distilled Models with vLLM

The full 671B DeepSeek-R1 model requires specialized infrastructure, but the **distilled variants** (such as the Qwen-based 32B and 14B models) run efficiently on vLLM. As noted in the repository documentation (README.md lines 68-70), vLLM is the primary recommended framework for these smaller, distilled versions.

### Launching the vLLM Server

Use the `vllm serve` command to start the OpenAI-compatible API server. The following example (documented in README.md lines 174-178) demonstrates serving the 32B distilled model with tensor parallelism:

```bash
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
    --tensor-parallel-size 2 \
    --max-model-len 32768 \
    --enforce-eager

```

**Key parameters explained:**

- **`--tensor-parallel-size 2`**: Distributes the model across 2 GPUs using tensor parallelism
- **`--max-model-len 32768`**: Sets the maximum sequence length (32,768 tokens for this distilled variant)
- **`--enforce-eager`**: Disables CUDA graph optimizations to avoid potential compatibility issues with MoE architectures

### Querying the Model

Once the server is running, interact with it via the OpenAI-compatible HTTP API at `http://localhost:8000/v1/completions` or `http://localhost:8000/v1/chat/completions`.

## Optimizing Inference Parameters

To achieve optimal reasoning quality when you run DeepSeek-R1 models locally using vLLM, adhere to the configuration guidelines in the repository (README.md lines 90-92):

- **Temperature**: Set between **0.5 and 0.7**, with **0.6** being the recommended default. Values outside this range may cause repetitive or incoherent reasoning outputs.
- **System Prompts**: **Do not use system prompts**. Embed all instructions directly within the user prompt to avoid interfering with the model's reasoning process.
- **Reasoning Extraction**: The model outputs reasoning content within `