DeepGEMM
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
Optimize BF16 and FP8 tensor memory layouts in DeepGEMM. Learn techniques for row-major alignment and TMA-compatible access on NVIDIA GPUs to boost performance.
Understanding the head_splits Parameter in fp8_gemm_nt_skip_head_mid for DeepGEMMLearn how the head_splits parameter in DeepGEMM's FP8 kernel efficiently handles sparse attention by inserting zero-padding and skipping masked computations.
How to Use DeepGEMM Weight Gradient Kernels for Training Backward PassesLearn how to use DeepGEMM weight gradient kernels for efficient backward passes in MoE training. Accelerate your deep learning with optimized GEMM operations.
fp8_gemm vs fp8_fp4_gemm in DeepGEMM: Architecture, Features, and Migration GuideExplore fp8_gemm vs fp8_fp4_gemm in DeepGEMM. Understand the core implementation supporting FP8 and FP4 formats on NVIDIA GPUs. Learn architecture, features, and migration.
How to Configure Block Size Multiples for Kernel Alignment in DeepGEMMConfigure block size multiples in DeepGEMM with set_block_size_multiple_of() to meet hardware alignment constraints and ensure optimal kernel performance.
DeepGEMM Scaling Factor Transformation Utilities: A Complete Guide to FP8 and FP4 QuantizationExplore DeepGEMM's scaling factor transformation utilities for FP8 FP4 quantization. Learn how these helpers convert FP32 tensors into hardware-optimized low-precision formats with quantization metadata.
DeepGEMM Recipe Parameter: How It Controls Kernel Selection and Tensor LayoutDiscover how the DeepGEMM recipe parameter controls CUDA kernel selection and tensor layout for FP8/FP4 matrix multiplication. Optimize your deep learning performance.
DeepGEMM CUDA Graph Compatibility for Masked Grouped GEMMs: Implementation GuideLearn how DeepGEMM achieves CUDA Graph compatibility for masked grouped GEMMs. Discover its stateless Triton kernel implementation for efficient graph capture.
What Is Programmatic Dependent Launch (PDL) in DeepGEMM and How to Enable ItLearn what Programmatic Dependent Launch PDL is in DeepGEMM and how to enable it. Dynamically compute launch parameters at Python runtime for enhanced kernel execution.
How to Handle NT/NN/TN/TT Transpose Conventions in DeepGEMMLearn to handle NT NN TN TT transpose conventions in DeepGEMM GEMM APIs. DeepGEMM requires explicit transposition for non-default layouts optimize performance.
Performance Implications of NVRTC vs NVCC Compilation in DeepGEMMDiscover NVRTC vs NVCC compilation performance in DeepGEMM. Understand latency differences and caching benefits for optimized deep learning.
How to Set Up Symmetric Memory Buffers for Multi-Process Mega MoE Execution in DeepGEMMLearn to set up symmetric memory buffers for multi-process Mega MoE execution in DeepGEMM. Use PyTorch's private API to allocate shared CUDA buffers and create input views for the fused kernel.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →