# DeepSeek-V3 | DeepSeek | Knowledge Base | Instagit

GitHub Stars: 102k

Repository: https://github.com/deepseek-ai/DeepSeek-V3

---

## Articles

### [Understanding Dynamic and Static Activation Quantization Schemes in DeepSeek-V3](/deepseek-ai/DeepSeek-V3/what-is-the-difference-between-dynamic-and-static-activation-quantization-schemes)

Explore dynamic vs static activation quantization. DeepSeek-V3 uses dynamic schemes for real-time scaling and peak accuracy. Understand how per-block scales improve inference.

- Tags: deep-dive
- Published: 2026-02-26

### [How to Set Up Distributed Inference with torchrun for Multi-Node Deployment](/deepseek-ai/DeepSeek-V3/how-to-set-up-distributed-inference-with-torchrun-for-multi-node-deployment)

Learn how to set up distributed inference with torchrun for multi-node deployment. Effortlessly launch generate.py using torchrun flags for efficient model scaling and sharded checkpoint loading across nodes.

- Tags: how-to-guide
- Published: 2026-02-26

### [DeepSeek-V3-Base vs DeepSeek-V3 Chat Model Architectures: Key Differences Explained](/deepseek-ai/DeepSeek-V3/what-are-the-differences-between-deepseek-v3-base-and-deepseek-v3-chat-model-architectures)

Explore the key differences between DeepSeek-V3-Base and DeepSeek-V3 Chat models. Understand their shared architecture and distinct fine-tuning for optimal performance.

- Tags: deep-dive
- Published: 2026-02-26

### [How weight_dequant Handles Per-Block Scaling in DeepSeek-V3 FP8 GEMM](/deepseek-ai/DeepSeek-V3/how-does-the-weight-dequant-function-handle-per-block-scaling-during-fp8-gemm)

Explore how weight_dequant manages per-block scaling for FP8 GEMM in DeepSeek-V3. Understand its custom Triton kernel for applying unique scaling factors to weight tiles.

- Tags: internals
- Published: 2026-02-26

### [How to Profile DeepSeek-V3 Inference Kernels Using Triton Autotuning](/deepseek-ai/DeepSeek-V3/how-to-profile-deepseek-v3-inference-kernels-using-triton-autotuning)

Profile DeepSeek-V3 inference kernels efficiently with Triton autotuning. Learn to capture runtime selections or benchmark specific kernel configurations for optimized performance.

- Tags: performance
- Published: 2026-02-26

### [How to Implement Custom Expert Routing Strategies in the DeepSeek-V3 Gate Module](/deepseek-ai/DeepSeek-V3/how-to-implement-custom-expert-routing-strategies-in-the-gate-module)

Master custom expert routing in DeepSeek-V3. Learn to modify hyperparameters, subclass the Gate class, or implement alternative scoring for advanced control.

- Tags: how-to-guide
- Published: 2026-02-26

### [DeepSeek-V3 MLA KV Cache Memory Footprint for 128K Context Length](/deepseek-ai/DeepSeek-V3/what-is-the-memory-footprint-of-kv-cache-for-128k-context-length-with-mla)

Discover the DeepSeek-V3 MLA KV cache memory footprint for 128K context. Learn about BFloat16 and FP8 quantization options impacting GPU memory usage, perfect for optimizing large context models.

- Tags: performance
- Published: 2026-02-26

### [How to Convert HuggingFace FP8 Weights to DeepSeek-V3 Format Using convert.py](/deepseek-ai/DeepSeek-V3/how-to-convert-huggingface-fp8-weights-to-custom-format-using-convert-py)

Learn to convert HuggingFace FP8 weights to DeepSeek-V3 format with convert.py. Reshape and shard checkpoints for DeepSeek-V3 inference engine.

- Tags: how-to-guide
- Published: 2026-02-26

### [DeepSeek-V3 quantization_config Parameters Explained: e4m3, weight_block_size, and activation_scheme](/deepseek-ai/DeepSeek-V3/what-do-the-quantization-config-parameters-e4m3-weight-block-size-activation-scheme-mean-in-config-json)

Understand DeepSeek-V3 quantization_config parameters like e4m3, weight_block_size, and activation_scheme to optimize LLM inference. Learn FP8 storage, block quantization, and activation scaling.

- Tags: deep-dive
- Published: 2026-02-26

### [How DeepSeek-V3 Handles Expert Load Imbalance in MoE Layers Without Auxiliary Loss](/deepseek-ai/DeepSeek-V3/how-does-the-model-handle-expert-load-imbalance-in-moe-layers-without-auxiliary-loss)

DeepSeek-V3 overcomes expert load imbalance in MoE layers using deterministic routing and group constraints without auxiliary loss. Learn how it optimizes performance.

- Tags: deep-dive
- Published: 2026-02-26

### [DeepSeek-V3 671B Model Parallelism Requirements: Tensor Parallel Size Setup](/deepseek-ai/DeepSeek-V3/what-are-the-model-parallelism-requirements-tensor-parallel-size-for-loading-671b-parameters)

Learn the tensor parallel size for DeepSeek-V3 671B model loading. Discover how to set up model parallelism across 16 GPUs for efficient inference.

- Tags: internals
- Published: 2026-02-26

### [DeepSeek-V3 RoPE Scaling Factor (`rope_factor=40`) and YaRN Extrapolation for 128K Context](/deepseek-ai/DeepSeek-V3/what-is-the-rope-scaling-factor-rope-factor-40-and-yarn-extrapolation-for-128k-context)

Discover how DeepSeek-V3 achieves 128K context with rope_factor=40 and YaRN extrapolation. Learn about frequency scaling and linear ramp correction for enhanced performance.

- Tags: deep-dive
- Published: 2026-02-26

### [How to Integrate DeepSeek-V3 with vLLM for Production-Grade Serving with Pipeline Parallelism](/deepseek-ai/DeepSeek-V3/how-to-integrate-deepseek-v3-with-vllm-for-production-grade-serving-with-pipeline-parallelism)

Integrate DeepSeek-V3 with vLLM for production-grade serving. Learn to convert checkpoints and configure pipeline parallelism for efficient deployment across multiple GPUs.

- Tags: how-to-guide
- Published: 2026-02-26

### [How to Debug OOM Errors When Running DeepSeek-V3 671B Parameter Model Inference](/deepseek-ai/DeepSeek-V3/how-to-debug-oom-errors-when-running-671b-parameter-model-inference)

Debug OOM errors for DeepSeek-V3 671B model inference by reducing sequence length and batch size, switching to bf16 dtype, and profiling memory layer by layer to pinpoint the issue.

- Tags: tutorial
- Published: 2026-02-26

### [Optimal Batch Size and Sequence Length Configurations for DeepSeek-V3 Inference on H800 GPUs](/deepseek-ai/DeepSeek-V3/what-are-the-optimal-batch-size-and-sequence-length-configurations-for-deepseek-v3-inference-on-h800-gpus)

Discover optimal batch size and sequence length for DeepSeek-V3 inference on H800 GPUs. Maximize performance with batch size 8 and sequence length 16,384 tokens.

- Tags: performance
- Published: 2026-02-26

### [How to Implement Speculative Decoding with DeepSeek-V3’s Multi-Token Prediction (MTP) Modules](/deepseek-ai/DeepSeek-V3/how-to-implement-speculative-decoding-using-the-multi-token-prediction-mtp-modules)

Discover how to implement speculative decoding with DeepSeek-V3s Multi Token Prediction MTP modules for faster inference. This guide explains how MTP modules draft and verify tokens, reducing latency.

- Tags: how-to-guide
- Published: 2026-02-26

### [How Block-wise FP8 Weight Quantization (128×128) Works in DeepSeek-V3](/deepseek-ai/DeepSeek-V3/how-does-block-wise-fp8-weight-quantization-128x128-work-in-the-kernel-implementation)

Discover how DeepSeek-V3 uses block-wise FP8 weight quantization (128x128) in its kernel. Learn about per-block scaling and on-the-fly de-quantization for efficient matrix multiplication.

- Tags: internals
- Published: 2026-02-26

### [KV Cache Eviction Strategy in MLA for Long Context Sequences: A Deep Dive into DeepSeek-V3](/deepseek-ai/DeepSeek-V3/what-is-the-kv-cache-eviction-strategy-in-mla-for-handling-long-context-sequences)

Discover DeepSeek-V3's KV cache eviction strategy using MLA to manage long context sequences. Learn how the implicit sliding-window automatically handles memory without explicit deletion. Optimized for efficiency.

- Tags: deep-dive
- Published: 2026-02-26

### [How to Configure Multi-Node Tensor Parallelism for DeepSeek-V3 Inference in SGLang](/deepseek-ai/DeepSeek-V3/how-to-configure-multi-node-tensor-parallelism-for-deepseek-v3-inference-in-sglang)

Learn to configure multi-node tensor parallelism for DeepSeek-V3 inference using SGLang. Easily scale your DeepSeek-V3 models across multiple machines with our guide.

- Tags: how-to-guide
- Published: 2026-02-26

### [How the DeepSeek-V3 MoE Gating Mechanism Routes Tokens Without Auxiliary Loss](/deepseek-ai/DeepSeek-V3/how-does-the-moe-gating-mechanism-route-tokens-to-experts-with-the-auxiliary-loss-free-strategy)

Discover how DeepSeek-V3 MoE gating routes tokens without auxiliary loss. Learn about its novel learned gating function, group filtering, and top-k selection for efficient expert allocation.

- Tags: internals
- Published: 2026-02-26

### [Naive vs Absorb Attention Implementations in DeepSeek-V3: MLA Optimization Guide](/deepseek-ai/DeepSeek-V3/what-is-the-difference-between-naive-and-absorb-attention-implementations-in-deepseek-v3)

Understand the difference between naive and absorb attention in DeepSeek-V3. Learn how absorb attention optimizes VRAM by compressing key-value caches for MLA.

- Tags: deep-dive
- Published: 2026-02-26

### [How Multi-Head Latent Attention (MLA) Reduces KV Cache Memory in DeepSeek-V3](/deepseek-ai/DeepSeek-V3/how-does-multi-head-latent-attention-mla-reduce-kv-cache-memory-compared-to-standard-mha)

Discover how DeepSeek-V3's Multi-Head Latent Attention (MLA) slashes KV cache memory by compressing keys and values reducing per-token storage from thousands to hundreds of floats.

- Tags: internals
- Published: 2026-02-26