# How to Implement Custom Expert Routing Strategies in the DeepSeek-V3 Gate Module

> Master custom expert routing in DeepSeek-V3. Learn to modify hyperparameters, subclass the Gate class, or implement alternative scoring for advanced control.

- Repository: [DeepSeek/DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3)
- Tags: how-to-guide
- Published: 2026-02-26

---

**You can customize expert routing in DeepSeek-V3 by modifying ModelArgs hyperparameters, subclassing the Gate class in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py), or replacing the linear projection with alternative scoring mechanisms like MLPs or hash-based routing.**

The **Gate** module in DeepSeek-V3 serves as the routing core for the Mixture-of-Experts (MoE) layer, determining which experts process each token. Located in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py) at lines 535-599, this component scores routed experts, selects top-k candidates, and returns routing weights and indices. Understanding how to implement custom expert routing strategies in this module allows researchers to experiment with novel MoE architectures without modifying the broader model infrastructure.

## Understanding the Gate Module Architecture

### Core Components in inference/model.py

The `Gate` class is defined between lines 535 and 599 in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py). During initialization, it reads hyper-parameters from the `ModelArgs` dataclass (lines 19-54) and creates the learnable routing matrix `weight` along with an optional `bias` parameter.

Key attributes include:
- `weight`: The linear projection matrix mapping hidden states to expert scores.
- `bias`: Optional bias added to raw scores before selection.
- `score_func`: Either `"softmax"` or `"sigmoid"`, determining how raw logits are normalized.
- `route_scale`: A scalar multiplier applied to final routing weights.

### The Forward Pass Flow

The `forward` method (lines 566-599) executes the routing logic in six distinct stages:

1. **Linear projection**: Computes raw scores via `linear(x, self.weight)`.
2. **Scoring function**: Applies `softmax` or `sigmoid` to obtain normalized probabilities.
3. **Bias addition**: Adds learned bias if initialized.
4. **Group-wise selection**: When `n_groups > 1`, reshapes scores to `[batch, groups, experts_per_group]`, selects top groups (`topk_groups`), and masks others.
5. **Top-k expert selection**: Picks the final `topk` experts per token using `torch.topk`.
6. **Weight normalization**: Normalizes sigmoid scores, scales by `route_scale`, and returns `(weights, indices)`.

## Method 1: Configuring Routing via ModelArgs

The simplest way to alter routing behavior is to modify the hyper-parameters passed to `ModelArgs` during model construction. These values automatically propagate into `Gate.__init__` and `Gate.forward`.

```python
from inference.model import ModelArgs, MoE

args = ModelArgs(
    n_routed_experts=64,
    n_activated_experts=4,      # Reduce from default for sparser routing

    n_expert_groups=2,            # Enable hierarchical group selection

    n_limited_groups=1,           # Select only 1 group before top-k

    score_func="sigmoid",         # Switch from softmax to sigmoid scoring

    route_scale=0.75,             # Dampen routing weight magnitudes

)
model = MoE(args)

# Verify the configuration

print(f"Gate score function: {model.gate.score_func}")
print(f"Top-k experts: {model.gate.topk}")

```

This approach requires no subclassing and is ideal for ablation studies on routing density and scoring functions.

## Method 2: Subclassing Gate for Advanced Customization

When linear projection proves insufficient, subclass `Gate` to inject complex scoring mechanisms while preserving the original grouping and top-k selection logic.

### Implementing an MLP-Based Scorer

Replace the single linear layer with a multi-layer perceptron to capture non-linear relationships between hidden states and expert affinity:

```python
import torch
import torch.nn as nn
from inference.model import Gate, ModelArgs, MoE

class MLPGate(Gate):
    def __init__(self, args: ModelArgs):
        super().__init__(args)
        # Replace linear projection with three-layer MLP

        self.scorer = nn.Sequential(
            nn.Linear(args.dim, args.dim // 2),
            nn.ReLU(),
            nn.Linear(args.dim // 2, args.n_routed_experts),
        )
        # Re-initialize bias to match new output dimension

        self.bias = nn.Parameter(torch.zeros(args.n_routed_experts))

    def forward(self, x: torch.Tensor):
        # Compute scores via MLP

        scores = self.scorer(x)
        
        # Apply scoring function

        if self.score_func == "softmax":
            scores = scores.softmax(dim=-1, dtype=torch.float32)
        else:
            scores = scores.sigmoid()
        
        original_scores = scores
        if self.bias is not None:
            scores = scores + self.bias
        
        # Re-use grouping and top-k logic from base implementation

        if self.n_groups > 1:
            scores = scores.view(x.size(0), self.n_groups, -1)
            if self.bias is None:
                group_scores = scores.amax(dim=-1)
            else:
                group_scores = scores.topk(2, dim=-1)[0].sum(dim=-1)
            indices = group_scores.topk(self.topk_groups, dim=-1)[1]
            mask = scores.new_ones(x.size(0), self.n_groups, dtype=bool).scatter_(1, indices, False)
            scores = scores.masked_fill_(mask.unsqueeze(-1), float("-inf")).flatten(1)
        
        # Final top-k selection and normalization

        indices = torch.topk(scores, self.topk, dim=-1)[1]
        weights = original_scores.gather(1, indices)
        if self.score_func == "sigmoid":
            weights = weights / weights.sum(dim=-1, keepdim=True)
        weights = weights * self.route_scale
        return weights.type_as(x), indices

# Usage

args = ModelArgs()
moe = MoE(args)
moe.gate = MLPGate(args)  # Plug in custom gate

x = torch.randn(12, args.dim)
out = moe(x)
print(out.shape)  # torch.Size([12, args.dim])

```

### Adding Deterministic Hash Routing

For experimental routing that bypasses learned parameters entirely, implement a hash-based router:

```python
import torch
import torch.nn as nn
from inference.model import MoE, ModelArgs

class HashGate(nn.Module):
    """
    Deterministic hash-based router that assigns tokens to experts
    based on token ID hash values rather than learned scores.
    """
    def __init__(self, args: ModelArgs):
        super().__init__()
        self.topk = args.n_activated_experts
        self.n_experts = args.n_routed_experts

    def forward(self, x: torch.Tensor):
        # Assume first column contains token identifiers

        token_ids = x[:, 0].long()
        expert_ids = torch.arange(self.n_experts, device=x.device)
        
        # Compute XOR hash between token and each expert

        hashes = (token_ids.unsqueeze(1) ^ expert_ids).float()
        
        # Select experts with lowest hash values (deterministic)

        indices = hashes.topk(self.topk, largest=False).indices
        weights = torch.full_like(indices, 1.0 / self.topk, dtype=x.dtype)
        return weights, indices

# Integration

args = ModelArgs()
moe = MoE(args)
moe.gate = HashGate(args)

x = torch.randn(5, args.dim)
out = moe(x)
print(out.shape)

```

## Method 3: Full Gate Replacement

When the routing algorithm diverges completely from the original top-k paradigm (e.g., reinforcement learning-based routing or k-means clustering), create a standalone module adhering to the interface contract `forward(x) -> (weights, indices)`:

```python
class ReinforcementLearningGate(nn.Module):
    def __init__(self, args: ModelArgs):
        super().__init__()
        self.policy_network = nn.Sequential(
            nn.Linear(args.dim, args.dim),
            nn.ReLU(),
            nn.Linear(args.dim, args.n_routed_experts)
        )
        self.topk = args.n_activated_experts
        
    def forward(self, x):
        logits = self.policy_network(x)
        # Sample or select based on policy

        weights, indices = torch.topk(torch.softmax(logits, dim=-1), self.topk)
        return weights, indices

```

The `MoE` class (lines 636-694 in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py)) consumes the Gate output via `self.gate(x)`, dispatching tokens to experts based solely on the returned indices and weights. This decoupling ensures that any object satisfying the return signature integrates seamlessly with the existing expert dispatch infrastructure.

## Summary

- The **Gate** module in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py) (lines 535-599) controls all routing decisions in DeepSeek-V3's MoE layer through a linear projection, scoring function, and top-k selection.
- **Hyper-parameter tuning** via `ModelArgs` offers the fastest customization path, adjusting `n_activated_experts`, `score_func`, and group routing settings without code changes.
- **Subclassing Gate** preserves the original grouping and normalization logic while allowing custom scoring mechanisms, such as MLP-based scorers or hash-based deterministic routing.
- **Full Gate replacement** is possible when implementing radically different algorithms (e.g., RL-based or clustering approaches), provided the new module returns `(weights, indices)` tensors.
- The `MoE` class (lines 636-694) remains agnostic to the specific routing implementation, consuming only the Gate's output tuple.

## Frequently Asked Questions

### What is the default routing strategy in DeepSeek-V3?

The default strategy uses a **learned linear projection** followed by either softmax or sigmoid scoring, then selects the top-k experts per token. When `n_expert_groups` is greater than 1, the Gate first selects the best groups using either max-scoring or top-2 aggregation before choosing final experts within those groups, as implemented in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py) lines 566-599.

### Can I use reinforcement learning to train the Gate module?

Yes. You can implement a custom `nn.Module` that replaces the standard Gate and includes a policy network trained via RL. Ensure your `forward` method returns a tuple of `(weights, indices)` compatible with the `MoE` class dispatch mechanism. The existing infrastructure in [`inference/model.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py) supports this because `MoE.forward` only depends on the Gate's output signature, not its internal training procedure.

### How does group-wise routing affect performance?

Group-wise routing reduces computational overhead by first selecting a subset of expert groups (controlled by `n_limited_groups`) before performing the final top-k selection within those groups. This hierarchical approach limits the number of experts evaluated during the second stage, improving inference speed at the potential cost of routing accuracy if the group selection is too restrictive.

### Where are the Gate weights stored in the checkpoint?

The Gate's linear projection weights and bias are mapped to specific tensor names during checkpoint conversion. You can verify the exact naming convention in [`inference/convert.py`](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/convert.py) (lines 20-25), which defines the mapping between the original training checkpoint keys and the inference model's parameter names used by the Gate module.