# What Base Model Is DeepSeek-R1 Based On?

> Discover which base model powers DeepSeek-R1. Learn how DeepSeek-R1 and DeepSeek-R1 Zero are fine-tuned from the DeepSeek-V3-Base model for optimal performance.

- Repository: [DeepSeek/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1)
- Tags: deep-dive
- Published: 2026-02-27

---

**DeepSeek-R1 and its DeepSeek-R1-Zero variant are fine-tuned from the DeepSeek-V3-Base model, as explicitly documented in the repository's model summary.**

Understanding what base model DeepSeek-R1 is based on is essential for researchers working to reproduce its reasoning capabilities or adapt its architecture for specialized tasks. The DeepSeek-AI team confirms that both model variants originate from the DeepSeek-V3-Base checkpoint, which provides the foundational weights and transformer architecture upon which the advanced reasoning behaviors are constructed.

## DeepSeek-V3-Base: The Pre-Trained Foundation

According to [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) in the DeepSeek-R1 repository, the model documentation explicitly states at lines 78–80 that both **DeepSeek-R1-Zero** and the full **DeepSeek-R1** are "trained based on DeepSeek-V3-Base"【README.md†L78-L80】. This establishes DeepSeek-V3-Base as the authoritative starting checkpoint, supplying the pre-trained parameter weights and Mixture-of-Experts (MoE) architecture before any reasoning-specific training begins. The base model contributes the extensive world knowledge, multilingual capabilities, and general linguistic patterns that serve as the substrate for the subsequent specialization process.

## From Base Model to Reasoning Specialist

The transformation from DeepSeek-V3-Base to DeepSeek-R1 involves applying **reinforcement learning (RL)** and **supervised fine-tuning (SFT)** on top of the frozen or warmed base weights. The accompanying research paper `DeepSeek_R1.pdf` details how the training pipeline builds upon the base model's representations to develop advanced chain-of-thought reasoning, mathematical problem-solving, and code generation capabilities【DeepSeek_R1.pdf】. For complete architectural specifications of the underlying transformer structure—including the 671 billion total parameters with 37 billion activated per token—developers must reference the separate [DeepSeek-V3 repository](https://github.com/deepseek-ai/DeepSeek-V3), which houses the definitive base model implementation.

## Loading DeepSeek-R1 from Hugging Face

When implementing the model in production environments, you load the fine-tuned DeepSeek-R1 weights rather than the raw DeepSeek-V3-Base checkpoint. The following Python implementation using the `transformers` library demonstrates proper initialization:

```python

# Example: loading DeepSeek-R1 from Hugging Face

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepseek-ai/DeepSeek-R1"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype="auto"
)

# Simple generation

prompt = "Explain the steps to solve a quadratic equation."
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=200, temperature=0.6)
print(tokenizer.decode(output[0], skip_special_tokens=True))

```

This instantiation automatically loads the architecture inherited from DeepSeek-V3-Base while applying the reasoning-optimized fine-tuning specific to DeepSeek-R1.

## Deploying Distilled Variants via vLLM

For high-throughput production serving of the smaller distilled variants—which inherit reasoning methodologies from the R1 training pipeline based on DeepSeek-V3-Base—the following vLLM command deploys the 32B parameter distilled model:

```bash

# Example: serving a distilled variant with vLLM

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --enforce-eager

```

Note that while distilled variants may utilize different architectural backbones (such as Qwen), they derive their reasoning training signal from the DeepSeek-R1 process that originated from the DeepSeek-V3-Base lineage.

## Summary

- **DeepSeek-V3-Base** serves as the foundational checkpoint for both DeepSeek-R1 and DeepSeek-R1-Zero, as documented in [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) lines 78–80.
- The base model provides the pre-trained weights and MoE architecture, while subsequent RL and SFT stages inject specialized reasoning capabilities.
- Production implementations require loading the final fine-tuned checkpoints rather than the base model to access the reasoning behaviors.
- Complete architectural details for the base model are maintained in the separate DeepSeek-V3 repository referenced by the DeepSeek-R1 documentation.

## Frequently Asked Questions

### Is DeepSeek-R1 built on DeepSeek-V3-Base?

Yes. According to the source code documentation in [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) at lines 78–80, both DeepSeek-R1 and DeepSeek-R1-Zero are explicitly trained based on the DeepSeek-V3-Base model. This relationship is fundamental to the model's ability to perform complex reasoning tasks.

### What is the difference between DeepSeek-R1 and DeepSeek-R1-Zero?

DeepSeek-R1-Zero is trained purely through large-scale reinforcement learning from DeepSeek-V3-Base without supervised fine-tuning data, while the full DeepSeek-R1 incorporates additional cold-start data and multi-stage training pipelines on top of the same base model weights.

### Can I access the DeepSeek-V3-Base weights separately?

Yes. The DeepSeek-V3-Base checkpoint is available in the separate [DeepSeek-V3 repository](https://github.com/deepseek-ai/DeepSeek-V3) maintained by DeepSeek-AI. These weights can be used for custom fine-tuning or research purposes, though they lack the reasoning optimization present in DeepSeek-R1.

### Where is the base model relationship documented?

The relationship is explicitly stated in the [`README.md`](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md) file within the DeepSeek-R1 repository at lines 78–80. Additional technical details regarding the training pipeline from base model to final reasoning model are provided in the `DeepSeek_R1.pdf` research paper included in the repository.