# fp8_gemm vs fp8_fp4_gemm in DeepGEMM: Architecture, Features, and Migration Guide

> Explore fp8_gemm vs fp8_fp4_gemm in DeepGEMM. Understand the core implementation supporting FP8 and FP4 formats on NVIDIA GPUs. Learn architecture, features, and migration.

- Repository: [DeepSeek/DeepGEMM](https://github.com/deepseek-ai/DeepGEMM)
- Tags: architecture
- Published: 2026-04-19

---

**`fp8_gemm` is a legacy compatibility alias that forwards to `fp8_fp4_gemm`, the core implementation that supports both FP8 and the memory-efficient FP4 (UE8M0) format on NVIDIA Hopper and Blackwell GPUs.**

In the DeepGEMM library, these two function families represent different layers of the API stack. While both execute high-performance matrix multiplication on quantized tensors, only `fp8_fp4_gemm` exposes the full feature set including packed FP4 scaling recipes and architecture-specific kernel selection.

## What is fp8_gemm?

`fp8_gemm` exists as a **thin backward-compatibility wrapper** that provides the original FP8-only API surface. According to the source code in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp), these symbols are explicitly aliased to their `fp8_fp4_gemm` counterparts at line 659:

```cpp
// csrc/apis/gemm.hpp (line 659)
m.attr("fp8_gemm_nt") = m.attr("fp8_fp4_gemm_nt");
m.attr("fp8_gemm_nn") = m.attr("fp8_fp4_gemm_nn");
// ... additional variants

```

When you call `fp8_gemm_nt`, the library immediately delegates to `fp8_fp4_gemm_nt` without additional logic. This design preserves existing codebases while funneling all execution through a unified implementation path.

## What is fp8_fp4_gemm?

`fp8_fp4_gemm` is the **core computational implementation** located at lines 50-100 of [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp). Unlike the legacy alias, this function handles both traditional FP8 quantization and the newer FP4 (packed UE8M0) format that reduces memory bandwidth on NVIDIA Hopper and Blackwell architectures.

The implementation performs three critical operations:

1. **Shape validation** for tiled matrix dimensions
2. **Scaling factor layout transformation** based on the `recipe` parameters
3. **Kernel dispatch** to architecture-specific JIT-compiled kernels

```cpp
// csrc/apis/gemm.hpp (simplified excerpt)
static void fp8_fp4_gemm_nt(
    const std::pair<torch::Tensor, torch::Tensor>& a,
    const std::pair<torch::Tensor, torch::Tensor>& b,
    const torch::Tensor& d,
    const std::optional<torch::Tensor>& c,
    std::optional<std::tuple<int, int, int>> recipe,
    std::optional<std::tuple<int, int>> recipe_a,
    std::optional<std::tuple<int, int>> recipe_b,
    const std::string& compiled_dims,
    const bool& disable_ue8m0_cast) {
    
    // Architecture-specific dispatch
    if (arch_major == 9 && sfa.scalar_type() == torch::kFloat) {
        sm90_fp8_gemm_1d1d(...);
    } else if (arch_major == 10 && sfa.scalar_type() == torch::kInt) {
        sm100_fp8_fp4_gemm_1d1d(...);
    }
}

```

The function selects between:
- **`sm90_fp8_gemm_1d1d`** from [`csrc/jit_kernels/impls/sm90_fp8_gemm_1d1d.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/jit_kernels/impls/sm90_fp8_gemm_1d1d.hpp) for Hopper (SM90) FP8-only mode
- **`sm100_fp8_fp4_gemm_1d1d`** from [`csrc/jit_kernels/impls/sm100_fp8_fp4_gemm_1d1d.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/jit_kernels/impls/sm100_fp8_fp4_gemm_1d1d.hpp) for Blackwell (SM100) with packed FP4 support

## Key Differences Between fp8_gemm and fp8_fp4_gemm

| Feature | `fp8_gemm` (Legacy) | `fp8_fp4_gemm` (Current) |
|---------|---------------------|--------------------------|
| **Implementation Layer** | Alias/wrapper in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp) line 659 | Core implementation in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp) lines 50-100 |
| **Format Support** | FP8 only (historically) | FP8 and packed FP4 (UE8M0) |
| **GPU Architecture** | SM90 (Hopper) via delegation | SM90 and SM100 (Blackwell) with native kernel selection |
| **API Parameters** | Standard tensor inputs | Additional `recipe`, `recipe_a`, `recipe_b`, and `disable_ue8m0_cast` flags |
| **Memory Bandwidth** | Standard FP8 traffic | Reduced traffic when using FP4 mode |
| **Deprecation Status** | Legacy alias for backward compatibility | Recommended API for new development |

## Code Examples: Calling Both APIs

The following examples demonstrate the functional equivalence of the legacy alias and the modern API, while highlighting the FP4-specific features available only in `fp8_fp4_gemm`.

```python
import torch
import deep_gemm as dg

# Common setup

M, N, K = 1024, 1024, 1024
a = torch.rand((M, K), dtype=torch.float16).cuda()
b = torch.rand((N, K), dtype=torch.float16).cuda()
c = torch.empty((M, N), dtype=torch.float16).cuda()

# -------------------------------------------------

# Legacy API: fp8_gemm_nt (backward compatibility)

# -------------------------------------------------

# Scaling factors for FP8 (float32)

sfa_fp8 = torch.ones(M, dtype=torch.float32).cuda()
sfb_fp8 = torch.ones(N, dtype=torch.float32).cuda()

# This call forwards internally to fp8_fp4_gemm_nt

dg.fp8_gemm_nt((a, sfa_fp8), (b, sfb_fp8), c)
print("Legacy API completed")

# -------------------------------------------------

# Modern API: fp8_fp4_gemm_nt (full features)

# -------------------------------------------------

# Case 1: Pure FP8 mode (same as above, but explicit)

dg.fp8_fp4_gemm_nt((a, sfa_fp8), (b, sfb_fp8), c)

# Case 2: FP4 packed mode (SM100/Blackwell only)

# Scaling factors packed as UE8M0 (uint8)

sfa_fp4 = torch.randint(0, 256, (M,), dtype=torch.uint8).cuda()
sfb_fp4 = torch.randint(0, 256, (N,), dtype=torch.uint8).cuda()

# Recipe describes how scaling factors are packed

recipe = (1, 1, 128)  # (gran_k_a, gran_k_b, gran_n)

dg.fp8_fp4_gemm_nt(
    (a, sfa_fp4), 
    (b, sfb_fp4), 
    c,
    recipe=recipe,
    compiled_dims="MKN"
)
print("Modern FP4 API completed")

```

When targeting NVIDIA Hopper (SM90), both APIs route to `sm90_fp8_gemm_1d1d` from [`csrc/jit_kernels/impls/sm90_fp8_gemm_1d1d.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/jit_kernels/impls/sm90_fp8_gemm_1d1d.hpp). On Blackwell (SM100), the modern API can dispatch to `sm100_fp8_fp4_gemm_1d1d` from [`csrc/jit_kernels/impls/sm100_fp8_fp4_gemm_1d1d.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/jit_kernels/impls/sm100_fp8_fp4_gemm_1d1d.hpp) when FP4 scaling is detected.

## Migration Guide: When to Use Which

**Use `fp8_fp4_gemm` for all new development.** This API provides complete access to DeepGEMM's capabilities, including the packed FP4 format that reduces memory bandwidth by storing scaling factors as 8-bit UE8M0 integers rather than 32-bit floats.

**Continue using `fp8_gemm` only for legacy code maintenance.** The function remains functional but offers no performance advantage. According to the binding code in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp), these symbols are explicitly aliased:

```cpp
m.attr("fp8_gemm_nt") = m.attr("fp8_fp4_gemm_nt");

```

**Enable FP4 mode when:**
- Running on SM100 (Blackwell) or newer architectures
- Memory bandwidth is the bottleneck in your GEMM operations
- You can represent scaling factors as packed UE8M0 format

**Stick to FP8 mode when:**
- Running on SM90 (Hopper) architecture
- You require floating-point scaling factors for numerical precision
- You have not yet implemented UE8M0 packing for your tensors

## Summary

- **`fp8_gemm`** is a legacy alias maintained for backward compatibility that forwards all calls to `fp8_fp4_gemm` without additional logic.
- **`fp8_fp4_gemm`** is the core implementation in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp) (lines 50-100) that handles both FP8 and the packed FP4 (UE8M0) format.
- The modern API accepts specialized parameters including `recipe`, `recipe_a`, `recipe_b`, and `disable_ue8m0_cast` to control FP4 packing behavior.
- Architecture-specific kernels are selected automatically: `sm90_fp8_gemm_1d1d` for Hopper (SM90) and `sm100_fp8_fp4_gemm_1d1d` for Blackwell (SM100).
- New code should use `fp8_fp4_gemm` directly to access full functionality and future-proof against removal of the legacy aliases.

## Frequently Asked Questions

### Is fp8_gemm deprecated in DeepGEMM?

While not officially removed, `fp8_gemm` functions are implemented as explicit aliases to their `fp8_fp4_gemm` counterparts in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp) (line 659). The DeepGEMM source code marks these as compatibility bindings, and new development should use the `fp8_fp4_gemm` variants to ensure access to all current and future features.

### Can I use fp8_fp4_gemm on older GPUs like A100?

No. The `fp8_fp4_gemm` implementation specifically checks for architecture compatibility in [`csrc/apis/gemm.hpp`](https://github.com/deepseek-ai/DeepGEMM/blob/main/csrc/apis/gemm.hpp). For SM90 (Hopper, H100), it dispatches to `sm90_fp8_gemm_1d1d`. For SM100 (Blackwell), it uses `sm100_fp8_fp4_gemm_1d1d`. The A100 (SM80) lacks native FP8 hardware support, and DeepGEMM does not provide kernels for this architecture.

### What is the FP4 (UE8M0) format and when should I use it?

FP4 refers to a packed 4-bit quantization format using UE8M0 (unsigned 8-bit magnitude, 0-bit exponent) storage for scaling factors. In `fp8_fp4_gemm`, enabling FP4 mode (by passing `int` scaling factors and a valid `recipe`) reduces memory bandwidth by packing scale factors into 8-bit integers rather than 32-bit floats. Use this mode on SM100 (Blackwell) GPUs when your workload is memory-bound and you can quantize your scaling factors to the UE8M0 representation.

### Do I need to change my code if I'm currently using fp8_gemm?

No immediate changes are required for functionality, as `fp8_gemm` calls will continue to work through the alias mechanism. However, to access FP4 capabilities and ensure forward compatibility, you should migrate your calls to `fp8_fp4_gemm`. The function signatures are compatible for FP8-only usage, requiring only the function name change in most cases. If you wish to enable FP4 mode, you will need to update your scaling factor tensors from `float32` to packed `uint8` format and provide the appropriate `recipe` tuple.