# How to Run DeepSeek-R1 Models Locally Using SGLang: A Complete Setup Guide

> Learn to run DeepSeek-R1 models locally with SGLang. This guide shows you how to set up the asynchronous inference server for high-throughput text generation via a local HTTP API.

- Repository: [DeepSeek/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1)
- Tags: how-to-guide
- Published: 2026-02-27

---

**Use SGLang's asynchronous inference server to serve DeepSeek-R1 checkpoints with tensor parallelism, exposing a local HTTP API for high-throughput generation.**

DeepSeek-R1 (including the Zero variant) are MoE-based large language models available as standard HuggingFace checkpoints. Because the Transformers library does not yet provide native support for these specific checkpoints, the recommended way to run DeepSeek-R1 models locally is to use **SGLang**, an inference engine capable of handling contexts up to 128 K tokens with efficient tensor-parallel execution.

## Prerequisites and Installation

### System Requirements

Running DeepSeek-R1 locally requires a Linux environment with Python ≥3.9, PyTorch, and sufficient GPU memory for the model size you select. The distilled variants (e.g., DeepSeek-R1-Distill-Qwen-32B) require less VRAM than the full 671B parameter model, but all variants benefit from multi-GPU setups via tensor parallelism.

### Installing SGLang

Install the SGLang package from PyPI. This provides the `sglang.launch_server` module used to start the inference backend.

```bash
pip install sglang

```

## Downloading the Model Checkpoint

SGLang can load models directly from HuggingFace Hub if you have internet connectivity, or you can pre-download the weights for offline use. For the 32B distilled variant used in the official examples:

```bash
git lfs install
git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

```

Alternatively, verify accessibility without cloning:

```bash
python -c "from huggingface_hub import hf_hub_download; \
print(hf_hub_download('deepseek-ai/DeepSeek-R1-Distill-Qwen-32B', 'config.json'))"

```

## Launching the SGLang Server

The DeepSeek-R1 repository's README documents the minimal command to start an SGLang server for these models. According to the source code at [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) (line 183), you must pass `--trust-remote-code` to handle the MoE architecture and `--tp` to specify tensor parallelism across GPUs.

```bash
python3 -m sglang.launch_server \
    --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
    --trust-remote-code \
    --tp 2

```

Adjust the `--tp` value to match your available GPU count. For example, use `--tp 4` on a 4-GPU machine to distribute the model layers evenly and increase throughput.

## Sending Inference Requests

Once the server is running on the default port (8000), send prompts via HTTP using the SGLang client library or raw REST calls. The following Python example demonstrates connecting to the local server and configuring generation parameters optimized for DeepSeek-R1's reasoning capabilities:

```python
import sglang as sgl

# Create a client pointing to the locally-running server

client = sgl.Client("http://localhost:8000")

prompt = """Please solve the following math problem step-by-step and give the final answer inside \\boxed{}:

What is the integral of sin(x)^2 from 0 to π?"""

# Generation settings: temperature between 0.5-0.7 yields optimal reasoning

response = client.generate(
    prompt,
    temperature=0.6,
    top_p=0.95,
    max_new_tokens=512,
    stop=None,
)

print("Model response:", response.text)

```

## Optimizing Performance for DeepSeek-R1

### Temperature and Sampling

According to the repository's recommendations, DeepSeek-R1 performs best when the **temperature is set between 0.5 and 0.7**. Values below 0.5 may produce repetitive reasoning traces, while values above 0.7 can degrade mathematical accuracy.

### Context Length Configuration

SGLang supports extremely long contexts up to 128 K tokens for DeepSeek-R1. Ensure your GPU memory is sufficient for the cache size when processing long documents, or reduce `--max-model-len` if you encounter out-of-memory errors during extended conversations.

## Summary

- **SGLang is the recommended backend** for running DeepSeek-R1 models locally because standard Transformers does not yet natively support these MoE checkpoints.
- **Use `--trust-remote-code`** when launching the server to properly initialize the model architecture.
- **Tensor parallelism via `--tp`** distributes the model across multiple GPUs, enabling larger batch sizes and faster inference.
- **Optimal generation parameters** include a temperature of 0.6 and top_p of 0.95 for mathematical reasoning tasks.

## Frequently Asked Questions

### What hardware is required to run DeepSeek-R1 locally?

You need a multi-GPU setup for the larger variants. The 32B distilled model runs comfortably on two modern GPUs with tensor parallelism (`--tp 2`), while the full 671B parameter model requires significantly more VRAM and GPUs. All variants require Python ≥3.9 and CUDA-capable hardware.

### Why does SGLang require the `--trust-remote-code` flag?

DeepSeek-R1 uses a Mixture-of-Experts (MoE) architecture with custom layer implementations not yet merged into the standard Transformers library. The `--trust-remote-code` parameter allows SGLang to execute the model-specific Python files included in the HuggingFace repository, ensuring correct weight loading and forward passes.

### How do I adjust the server for different model sizes?

Change the `--model` path to point to any DeepSeek-R1 checkpoint (e.g., `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`) and modify `--tp` to match your GPU count. The launch command structure remains identical regardless of model size, though you must ensure you have sufficient aggregate GPU memory for the chosen checkpoint.

### Can I use the server without the Python client library?

Yes. The SGLang server exposes a standard OpenAI-compatible HTTP API at `http://localhost:8000/v1/chat/completions`. You can send POST requests with JSON payloads containing the prompt, temperature, and max_tokens fields using `curl`, JavaScript, or any HTTP client.