What Base Model Is DeepSeek-R1 Based On?

DeepSeek-R1 and its DeepSeek-R1-Zero variant are fine-tuned from the DeepSeek-V3-Base model, as explicitly documented in the repository's model summary.

Understanding what base model DeepSeek-R1 is based on is essential for researchers working to reproduce its reasoning capabilities or adapt its architecture for specialized tasks. The DeepSeek-AI team confirms that both model variants originate from the DeepSeek-V3-Base checkpoint, which provides the foundational weights and transformer architecture upon which the advanced reasoning behaviors are constructed.

DeepSeek-V3-Base: The Pre-Trained Foundation

According to README.md in the DeepSeek-R1 repository, the model documentation explicitly states at lines 78–80 that both DeepSeek-R1-Zero and the full DeepSeek-R1 are "trained based on DeepSeek-V3-Base"【README.md†L78-L80】. This establishes DeepSeek-V3-Base as the authoritative starting checkpoint, supplying the pre-trained parameter weights and Mixture-of-Experts (MoE) architecture before any reasoning-specific training begins. The base model contributes the extensive world knowledge, multilingual capabilities, and general linguistic patterns that serve as the substrate for the subsequent specialization process.

From Base Model to Reasoning Specialist

The transformation from DeepSeek-V3-Base to DeepSeek-R1 involves applying reinforcement learning (RL) and supervised fine-tuning (SFT) on top of the frozen or warmed base weights. The accompanying research paper DeepSeek_R1.pdf details how the training pipeline builds upon the base model's representations to develop advanced chain-of-thought reasoning, mathematical problem-solving, and code generation capabilities【DeepSeek_R1.pdf】. For complete architectural specifications of the underlying transformer structure—including the 671 billion total parameters with 37 billion activated per token—developers must reference the separate DeepSeek-V3 repository, which houses the definitive base model implementation.

Loading DeepSeek-R1 from Hugging Face

When implementing the model in production environments, you load the fine-tuned DeepSeek-R1 weights rather than the raw DeepSeek-V3-Base checkpoint. The following Python implementation using the transformers library demonstrates proper initialization:

# Example: loading DeepSeek-R1 from Hugging Face

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepseek-ai/DeepSeek-R1"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    trust_remote_code=True,
    torch_dtype="auto"
)

# Simple generation

prompt = "Explain the steps to solve a quadratic equation."
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=200, temperature=0.6)
print(tokenizer.decode(output[0], skip_special_tokens=True))

This instantiation automatically loads the architecture inherited from DeepSeek-V3-Base while applying the reasoning-optimized fine-tuning specific to DeepSeek-R1.

Deploying Distilled Variants via vLLM

For high-throughput production serving of the smaller distilled variants—which inherit reasoning methodologies from the R1 training pipeline based on DeepSeek-V3-Base—the following vLLM command deploys the 32B parameter distilled model:

# Example: serving a distilled variant with vLLM

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --enforce-eager

Note that while distilled variants may utilize different architectural backbones (such as Qwen), they derive their reasoning training signal from the DeepSeek-R1 process that originated from the DeepSeek-V3-Base lineage.

Summary

• DeepSeek-V3-Base serves as the foundational checkpoint for both DeepSeek-R1 and DeepSeek-R1-Zero, as documented in README.md lines 78–80.
• The base model provides the pre-trained weights and MoE architecture, while subsequent RL and SFT stages inject specialized reasoning capabilities.
• Production implementations require loading the final fine-tuned checkpoints rather than the base model to access the reasoning behaviors.
• Complete architectural details for the base model are maintained in the separate DeepSeek-V3 repository referenced by the DeepSeek-R1 documentation.

Frequently Asked Questions

Is DeepSeek-R1 built on DeepSeek-V3-Base?

Yes. According to the source code documentation in README.md at lines 78–80, both DeepSeek-R1 and DeepSeek-R1-Zero are explicitly trained based on the DeepSeek-V3-Base model. This relationship is fundamental to the model's ability to perform complex reasoning tasks.

What is the difference between DeepSeek-R1 and DeepSeek-R1-Zero?

DeepSeek-R1-Zero is trained purely through large-scale reinforcement learning from DeepSeek-V3-Base without supervised fine-tuning data, while the full DeepSeek-R1 incorporates additional cold-start data and multi-stage training pipelines on top of the same base model weights.

Can I access the DeepSeek-V3-Base weights separately?

Yes. The DeepSeek-V3-Base checkpoint is available in the separate DeepSeek-V3 repository maintained by DeepSeek-AI. These weights can be used for custom fine-tuning or research purposes, though they lack the reasoning optimization present in DeepSeek-R1.

Where is the base model relationship documented?

The relationship is explicitly stated in the README.md file within the DeepSeek-R1 repository at lines 78–80. Additional technical details regarding the training pipeline from base model to final reasoning model are provided in the DeepSeek_R1.pdf research paper included in the repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}