# Understanding Dynamic and Static Activation Quantization Schemes in DeepSeek-V3

> Explore dynamic vs static activation quantization. DeepSeek-V3 uses dynamic schemes for real-time scaling and peak accuracy. Understand how per-block scales improve inference.

- Repository: [DeepSeek/DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3)
- Tags: deep-dive
- Published: 2026-02-26

---

**Dynamic activation quantization computes scaling factors at runtime based on actual activation values, while static activation quantization uses pre-computed scales determined during model calibration.** DeepSeek-V3 exclusively implements the dynamic scheme, calculating per-block scales on-the-fly during inference for maximum accuracy.

Activation quantization reduces intermediate tensor precision from full-precision formats like FP32 to low-bit representations such as FP8. The choice between **dynamic and static activation quantization schemes** fundamentally determines how scaling factors map floating-point values to quantized integers. According to the DeepSeek-V3 source code, the repository implements dynamic quantization through kernel-level functions that adapt to real-time input distributions.

## How Dynamic Activation Quantization Works

Dynamic quantization calculates scaling factors for every activation block during inference based on the actual runtime data. In DeepSeek-V3, this is handled by the `act_quant` function in [`inference/kernel.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/kernel.py) (lines 24-33), which processes contiguous tensors and derives scales from the maximum absolute values found in each block.

The implementation uses a default **block size of 128 channels**, computing the scale as `s = amax / 448` where `amax` is the clamped maximum absolute value. When `scale_fmt` is set to `"ue8m0"`, the function optionally rounds scales to power-of-two values for hardware compatibility. The `linear` wrapper in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py) (lines 159-166) calls `act_quant` during forward passes, integrating dynamic quantization into the generation pipeline.

```python
import torch
from inference.kernel import act_quant

# x: a contiguous tensor whose last dimension is a multiple of block_size (default 128)

x = torch.randn(1, 1024, dtype=torch.float32)

# Quantize activations on the fly

q_x, scale = act_quant(x, block_size=128, scale_fmt=None)

# q_x has dtype torch.float8_e4m3fn, scale holds per‑block FP32 scales

```

## How Static Activation Quantization Differs

Static quantization relies on scales computed ahead of time, typically during model preparation or calibration on representative datasets. These pre-determined scales are stored alongside weights in the checkpoint files and applied uniformly at inference time, eliminating the need for per-input computation.

Unlike the dynamic approach, static schemes do not require the `act_quant` function to analyze activation statistics during forward passes. Instead, the model loads `static_scales` tensors and applies them directly to dequantize values. While this reduces runtime overhead, it cannot adapt when input distributions differ from the calibration data.

```python

# Assume `static_scales` is a tensor saved with the model checkpoint

# and `static_dequant` simply divides by that scale without recomputing it.

def static_dequant(q_x, static_scales):
    return q_x.float() * static_scales.unsqueeze(-1)

# Load q_x from checkpoint and apply the precomputed scale

dequant_x = static_dequant(q_x, static_scales)

```

## Key Differences Between Dynamic and Static Schemes

**Scale Computation Timing:** Dynamic schemes compute `s = amax / 448` during inference for every block, while static schemes load pre-computed values from disk.

**Memory and Latency Overhead:** Dynamic quantization requires passing per-block scale tensors between layers, adding small memory reads and computational overhead. Static quantization minimizes runtime work by reusing stored scales but sacrifices adaptability.

**Accuracy Characteristics:** Dynamic quantization adapts to the current input distribution, often yielding higher accuracy when activation statistics vary across different generation steps. Static quantization performs best when inference data matches the calibration distribution exactly.

**Implementation Status:** DeepSeek-V3 exclusively implements dynamic quantization. The `quantization_config` in the released FP8 weight files explicitly declares `"activation_scheme": "dynamic"` as documented in [`README_WEIGHTS.md`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md) (lines 66-73). Static scheme code paths that bypass `act_quant` are not present in the current source.

## Summary

- **Dynamic quantization** computes per-block scales at runtime via `act_quant` in [`inference/kernel.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/kernel.py), offering superior adaptability to input distribution shifts.
- **Static quantization** applies pre-calibrated scales stored with model weights, minimizing computational overhead but requiring distribution stability.
- DeepSeek-V3 adopts the **dynamic scheme exclusively** for its FP8 inference, using 128-channel blocks and the `torch.float8_e4m3fn` format.
- The configuration schema supports `"activation_scheme": "static"`, though the corresponding implementation path is not included in the current release.

## Frequently Asked Questions

### What is the primary advantage of dynamic activation quantization?

Dynamic quantization adapts to the current input distribution by computing scales on-the-fly, often yielding higher accuracy when activation statistics vary across different inputs or generation steps. This flexibility comes at the cost of small per-inference computational overhead for calculating `amax` values.

### Why does DeepSeek-V3 use FP8 E4M3FN for quantized activations?

The E4M3FN format provides 4 exponent bits and 3 mantissa bits with finite normalization, offering a better trade-off between range and precision for neural network activations compared to other 8-bit floating-point variants. The `act_quant` function in [`inference/kernel.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/kernel.py) returns tensors with this specific dtype.

### How does the block size parameter affect quantization accuracy?

The default block size of 128 channels determines the granularity of scale computation, where each block receives its own scale factor derived from `amax / 448`. Smaller blocks increase granularity and potentially improve accuracy but require more memory for storing per-block scales, while larger blocks reduce overhead at the cost of quantization fidelity.

### Can I use static activation quantization with DeepSeek-V3 weights?

The current release does not provide static quantization implementations. While the configuration schema in [`README_WEIGHTS.md`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md) allows specifying `"activation_scheme": "static"`, the source code lacks the necessary code paths to load and apply pre-computed activation scales, making dynamic quantization the only functional option.