# DeepGEMM | DeepSeek | Knowledge Base | Instagit

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

GitHub Stars: 6.6k

Repository: https://github.com/deepseek-ai/DeepGEMM

---

## Articles

### [How to Optimize Memory Layout for BF16 vs FP8 Tensors in DeepGEMM](/deepseek-ai/DeepGEMM/optimize-memory-layout-bf16-vs-fp8-tensors-deepgemm)

Optimize BF16 and FP8 tensor memory layouts in DeepGEMM. Learn techniques for row-major alignment and TMA-compatible access on NVIDIA GPUs to boost performance.

- Tags: performance
- Published: 2026-04-19

### [Understanding the head_splits Parameter in fp8_gemm_nt_skip_head_mid for DeepGEMM](/deepseek-ai/DeepGEMM/deepgemm-fp8_gemm_nt_skip_head_mid-head_splits-parameter)

Learn how the head_splits parameter in DeepGEMM's FP8 kernel efficiently handles sparse attention by inserting zero-padding and skipping masked computations.

- Tags: deep-dive
- Published: 2026-04-19

### [How to Use DeepGEMM Weight Gradient Kernels for Training Backward Passes](/deepseek-ai/DeepGEMM/how-to-use-deepgemm-weight-gradient-kernels-training-backward-passes)

Learn how to use DeepGEMM weight gradient kernels for efficient backward passes in MoE training. Accelerate your deep learning with optimized GEMM operations.

- Tags: how-to-guide
- Published: 2026-04-19

### [fp8_gemm vs fp8_fp4_gemm in DeepGEMM: Architecture, Features, and Migration Guide](/deepseek-ai/DeepGEMM/deepgemm-fp8_gemm-vs-fp8_fp4_gemm-functions)

Explore fp8_gemm vs fp8_fp4_gemm in DeepGEMM. Understand the core implementation supporting FP8 and FP4 formats on NVIDIA GPUs. Learn architecture, features, and migration.

- Tags: architecture
- Published: 2026-04-19

### [How to Configure Block Size Multiples for Kernel Alignment in DeepGEMM](/deepseek-ai/DeepGEMM/deepgemm-configure-block-size-multiples-kernel-alignment)

Configure block size multiples in DeepGEMM with set_block_size_multiple_of() to meet hardware alignment constraints and ensure optimal kernel performance.

- Tags: how-to-guide
- Published: 2026-04-19

### [DeepGEMM Scaling Factor Transformation Utilities: A Complete Guide to FP8 and FP4 Quantization](/deepseek-ai/DeepGEMM/deepgemm-scaling-factor-transformation-utilities)

Explore DeepGEMM's scaling factor transformation utilities for FP8 FP4 quantization. Learn how these helpers convert FP32 tensors into hardware-optimized low-precision formats with quantization metadata.

- Tags: tutorial
- Published: 2026-04-19

### [DeepGEMM Recipe Parameter: How It Controls Kernel Selection and Tensor Layout](/deepseek-ai/DeepGEMM/deepgemm-recipe-parameter-kernel-selection)

Discover how the DeepGEMM recipe parameter controls CUDA kernel selection and tensor layout for FP8/FP4 matrix multiplication. Optimize your deep learning performance.

- Tags: internals
- Published: 2026-04-19

### [DeepGEMM CUDA Graph Compatibility for Masked Grouped GEMMs: Implementation Guide](/deepseek-ai/DeepGEMM/deepgemm-cuda-graph-compatibility-masked-grouped-gemms)

Learn how DeepGEMM achieves CUDA Graph compatibility for masked grouped GEMMs. Discover its stateless Triton kernel implementation for efficient graph capture.

- Tags: how-to-guide
- Published: 2026-04-19

### [What Is Programmatic Dependent Launch (PDL) in DeepGEMM and How to Enable It](/deepseek-ai/DeepGEMM/deepgemm-programmatic-dependent-launch-pdl)

Learn what Programmatic Dependent Launch PDL is in DeepGEMM and how to enable it. Dynamically compute launch parameters at Python runtime for enhanced kernel execution.

- Tags: deep-dive
- Published: 2026-04-19

### [How to Handle NT/NN/TN/TT Transpose Conventions in DeepGEMM](/deepseek-ai/DeepGEMM/deepgemm-gemm-api-transpose-conventions)

Learn to handle NT NN TN TT transpose conventions in DeepGEMM GEMM APIs. DeepGEMM requires explicit transposition for non-default layouts optimize performance.

- Tags: how-to-guide
- Published: 2026-04-19

### [Performance Implications of NVRTC vs NVCC Compilation in DeepGEMM](/deepseek-ai/DeepGEMM/deepgemm-nvrtc-vs-nvcc-compilation-performance)

Discover NVRTC vs NVCC compilation performance in DeepGEMM. Understand latency differences and caching benefits for optimized deep learning.

- Tags: performance
- Published: 2026-04-19

### [How to Set Up Symmetric Memory Buffers for Multi-Process Mega MoE Execution in DeepGEMM](/deepseek-ai/DeepGEMM/setup-symmetric-memory-buffers-multi-process-mega-moe)

Learn to set up symmetric memory buffers for multi-process Mega MoE execution in DeepGEMM. Use PyTorch's private API to allocate shared CUDA buffers and create input views for the fused kernel.

- Tags: how-to-guide
- Published: 2026-04-19

### [UE8M0 Format in DeepGEMM: How SM100 Accelerates FP8 GEMM with Packed Scaling Factors](/deepseek-ai/DeepGEMM/deepgemm-ue8m0-format-scaling-factors-sm100)

Discover how DeepGEMM leverages UE8M0 format to pack scaling factors, enabling SM100 GPUs to accelerate FP8 GEMM computations directly on tensor cores without conversion overhead.

- Tags: deep-dive
- Published: 2026-04-19

### [How to Use K-Grouped GEMM for MoE Weight Backward Pass in DeepGEMM](/deepseek-ai/DeepGEMM/how-to-use-k-grouped-gemm-moe-weight-backward-pass)

Learn how K-grouped GEMM speeds up MoE weight backward pass in DeepGEMM. This method efficiently calculates expert gradients with batched kernel launches for better performance.

- Tags: how-to-guide
- Published: 2026-04-19

### [1D1D vs 1D2D GEMM Kernels in DeepGEMM: Architecture and Usage Guide](/deepseek-ai/DeepGEMM/deepgemm-1d1d-vs-1d2d-gemm-kernels)

Understand 1D1D vs 1D2D GEMM kernels in DeepGEMM. Learn when to use each for optimal FP8 GEMM performance with standard, grouped, or masked matrices.

- Tags: architecture
- Published: 2026-04-19

### [How to Debug JIT Compilation Issues in DeepGEMM Using DG_JIT_DEBUG and DG_PRINT_CONFIGS](/deepseek-ai/DeepGEMM/how-to-debug-deepgemm-jit-compilation-issues)

Debug DeepGEMM JIT compilation issues by enabling DG_JIT_DEBUG to view NVCC/NVRTC commands and DG_PRINT_CONFIGS to log GEMM configurations. Resolve performance bottlenecks efficiently.

- Tags: how-to-guide
- Published: 2026-04-19

### [TMA Alignment Requirement for DeepGEMM Tensors: 16-Byte Hardware Constraint Explained](/deepseek-ai/DeepGEMM/deepgemm-tma-alignment-requirement-for-tensors)

Understand the TMA alignment requirement for DeepGEMM tensors. Learn about the 16-byte hardware constraint and how it's enforced in the C++ backend accessed via check sf layout.

- Tags: deep-dive
- Published: 2026-04-19

### [How to Implement MQA (Multi-Query Attention) Logits Computation with DeepGEMM](/deepseek-ai/DeepGEMM/how-to-implement-mqa-logits-computation-deepgemm)

Learn how to implement MQA logits computation with DeepGEMM. Discover optimized CUDA kernels leveraging TMA, UMMA, and pipelining on NVIDIA SM100. Accelerate your AI models.

- Tags: how-to-guide
- Published: 2026-04-19

### [How Mega MoE Overlaps Expert Parallelism Communication with Computation in DeepGEMM](/deepseek-ai/DeepGEMM/mega-moe-overlap-expert-parallelism-communication-computation)

Learn how Mega MoE overlaps expert parallelism communication with computation. Discover techniques that hide data movement using symmetric memory buffers and asynchronous NCCL streams.

- Tags: internals
- Published: 2026-04-19

### [Contiguous and Masked Grouped GEMM Layouts in DeepGEMM: When to Use Each](/deepseek-ai/DeepGEMM/deepgemm-contiguous-vs-masked-grouped-gemm-layouts)

Learn when to use contiguous vs masked grouped GEMM layouts in DeepGEMM. Optimize performance for uniform or variable batched workloads with clear guidance on layout selection.

- Tags: deep-dive
- Published: 2026-04-19

### [How to Use FP8 and FP4 GEMM Kernels with Proper Scaling Factor Layouts in DeepGEMM](/deepseek-ai/DeepGEMM/how-to-use-fp8-fp4-gemm-kernels-with-scaling-factor-layouts)

Master FP8 and FP4 GEMM kernels in DeepGEMM by transforming scaling factors using transform_sf_pair_into_required_layout. Learn the correct layouts for SM 9.0 and SM 10.0.

- Tags: how-to-guide
- Published: 2026-04-19

### [SM90 vs SM100 Architecture Support in DeepGEMM: Hopper vs Ada Differences](/deepseek-ai/DeepGEMM/deepgemm-sm90-vs-sm100-architecture-support)

Discover the SM90 vs SM100 architecture support differences in DeepGEMM. Learn about Hopper's NT layout with FP32 scaling versus Ada's comprehensive layout support with packed UE8M0 scaling.

- Tags: deep-dive
- Published: 2026-04-19

### [How DeepGEMM JIT Compilation Works: Environment Variables and Implementation](/deepseek-ai/DeepGEMM/how-does-deepgemm-jit-compilation-work-and-environment-variables)

Explore how DeepGEMM JIT compilation creates GPU kernels from Python at runtime. Discover environment variables like DG_JIT_ to control this process.

- Tags: internals
- Published: 2026-04-19