# How Multi-Lane Hyper-Connections and Sinkhorn Iteration Work in Needle 2

> Learn how Needle 2 utilizes multi-lane hyper-connections and Sinkhorn iteration for efficient signal redistribution and mass preservation in transformer layers. Understand its parallel processing and learned gating.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: internals
- Published: 2026-08-25

---

**Multi-lane hyper-connections split transformer layers into parallel "lanes" with learned gating and Sinkhorn-based residual mixing, while Sinkhorn iteration generates doubly-stochastic routing matrices that redistribute signals across lanes in a mass-preserving, differentiable way.**

Needle 2's Simple Attention Network replaces the traditional feed-forward layer with a Hadamard-MLP and introduces these two mechanisms to enable adaptive, efficient routing across parallel processing paths. This article explains exactly how these components function in the source code.

## What Are Multi-Lane Hyper-Connections?

**Multi-lane hyper-connections (MHC)** partition each transformer layer into `mhc_lanes` independent pathways that process information in parallel. Unlike standard residual connections, these lanes interact through three distinct gating stages, allowing the network to learn dynamic information flow.

The mechanism resides in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), with the core implementation split across two locations:

- **Lane broadcasting** occurs in `Stack.__call__` at lines 93-94
- **Per-layer parameters** are defined inside `_ScanBody` in the `hc` dictionary at lines 68-73

### The Five-Stage Pipeline

| Stage | Operation | Code Location |
|-------|-----------|---------------|
| Lane broadcasting | Expand `x` from `[B, T, d_model]` to `[B, T, n, d_model]` | `Stack.__call__`, lines 94-95 |
| Pre-gate (`hpre`) | Project and gate engram values before injection | `_ScanBody`, `phi_pre`, `b_pre` |
| Post-gate (`hpost`) | Scale block outputs before reintegration | `_ScanBody`, `a_post`, `b_post` |
| Residual mixing (`hres`) | Sinkhorn-based lane blending | `_ScanBody`, `phi_res` → `_sinkhorn` |
| Lane aggregation | Sum over lane dimension to restore original shape | Final operation in scan body |

### Pre-Gate Mechanism

The pre-gate controls how much engram information enters each lane. RMS-normalized embeddings `nx` pass through a learned linear projection `phi_pre`, followed by a sigmoid with bias `b_pre` and offset `pre_off`:

```

hpre = sigmoid(nx @ phi_pre + b_pre + pre_off)

```

This produces per-lane gating weights that scale engram values before they merge with the token stream.

### Post-Gate Mechanism

After the main `Block` computes its update `y`, the post-gate determines reintegration strength using learned scalars `a_post`, `b_post`, and offset `post_off`. This creates adaptive residual pathways that vary by lane and layer depth.

### Residual Mixing with Sinkhorn

The residual term `res = nx @ phi_res` becomes a routing matrix through the Sinkhorn operator. The resulting `hres` softly blends lane-wise values (`xf`) before combination with the gated update, enabling cross-lane information redistribution without hard routing decisions.

## How Sinkhorn Iteration Enables Differentiable Routing

The **Sinkhorn iteration** in Needle 2 solves a critical problem: how to mix lane representations with guaranteed constraints while maintaining differentiability. The private helper `_sinkhorn` at [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py) lines 69-74 implements this via log-space normalization.

### The Iteration Algorithm

For `iters` iterations (default 20), the algorithm alternates row and column normalization in log-space:

```python
log_K = log_K - jax.nn.logsumexp(log_K, axis=-1, keepdims=True)  # row normalization

log_K = log_K - jax.nn.logsumexp(log_K, axis=-2, keepdims=True)  # column normalization

```

Exponentiating the result yields a **doubly-stochastic matrix** where every row and column sums to exactly 1.

### Why Doubly-Stochastic Constraints Matter

This mathematical property ensures three critical behaviors:

1. **Mass preservation** — No lane arbitrarily amplifies or attenuates total signal magnitude
2. **Complete assignment** — All residual information is redistributed, none is lost
3. **Differentiability** — Gradients flow through the entire routing operation for end-to-end learning

The constraint mirrors optimal transport theory, where Sinkhorn iteration provides an efficient approximation to the optimal assignment problem.

## Implementation Walkthrough

### Activating Multi-Lane Hyper-Connections

Configure `mhc_lanes` in the transformer config to enable the mechanism:

```python
import jax.numpy as jnp
from needle.model.architecture import TransformerConfig, SimpleAttentionNetwork

# Build model with 4 hyper-connection lanes

cfg = TransformerConfig(
    d_model=768,
    num_heads=12,
    num_kv_heads=6,
    num_layers=27,
    mhc_lanes=4,          # activates multi-lane hyper-connections

    engram_layers=(2, 15),
)

model = SimpleAttentionNetwork(cfg)

# Forward pass automatically uses MHC

tokens = jnp.arange(0, 16)[None, :]   # shape (1, 16)

logits = model(tokens)                # shape (1, 16, vocab_size)

print("Output shape:", logits.shape)

```

The `mhc_lanes=4` parameter triggers:
- Creation of `phi_pre`, `phi_post`, `phi_res` linear maps per layer
- Initialization of `a_post`, `b_pre`, `b_post` gating scalars
- Integration of the Sinkhorn operator into the forward pass

### Direct Sinkhorn Invocation

For debugging or custom routing, access the operator directly:

```python
from needle.model.architecture import _sinkhorn

# Random routing logits: batch=1, 8 lanes mixing to 8 lanes

raw_logits = jnp.ones((1, 8, 8)) * 0.3

# Generate doubly-stochastic mixing matrix

mixing_matrix = _sinkhorn(raw_logits, iters=20)

# Verify constraints

print("Row sums:", mixing_matrix.sum(axis=-1))    # all 1.0

print("Column sums:", mixing_matrix.sum(axis=-2)) # all 1.0

```

The output matrix can replace or augment standard attention patterns in specialized architectures.

## Architecture Files and Their Roles

| File | Purpose |
|------|---------|
| [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) | Core implementation: `SimpleAttentionNetwork`, `_ScanBody`, `_sinkhorn`, and all hyper-connection parameters |
| [`needle/model/decode.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/decode.py) | Weight deserialization mapping saved checkpoints to `mhc_*` tensors |
| [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) | Checkpoint packaging including hyper-connection state |
| [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) | High-level architecture description with hyper-connection rationale |

## Summary

- **Multi-lane hyper-connections** split layers into parallel pathways with three-stage gating (pre-gate, post-gate, residual mixing), implemented in `Stack.__call__` and `_ScanBody` within [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py)

- **Sinkhorn iteration** at `_sinkhorn` (architecture.py lines 69-74) produces differentiable doubly-stochastic matrices via alternating log-space normalization, enabling mass-preserving lane mixing

- The combination replaces rigid residual connections with learned, adaptive routing that allocates computational capacity dynamically across lanes

- All mechanisms are fully differentiable and trained end-to-end with standard gradient descent

## Frequently Asked Questions

### How many Sinkhorn iterations does Needle 2 use by default?

The default is **20 iterations** as specified in the `_sinkhorn` function signature. This balances convergence accuracy against computational overhead—empirically sufficient for stable doubly-stochastic approximation without excessive latency in the forward pass.

### Can I change the number of lanes after training?

No. The `mhc_lanes` parameter determines the shape of learned tensors (`phi_pre`, `phi_post`, `phi_res`, and all gating scalars) throughout the network. Changing this requires reinitializing these parameters and retraining. The [`decode.py`](https://github.com/cactus-compute/needle/blob/main/decode.py) file maps saved weights to these specific shapes during checkpoint loading.

### Why use Sinkhorn instead of softmax for routing?

**Softmax** normalizes only across rows, permitting arbitrary column sums that can amplify or suppress total signal magnitude. **Sinkhorn** enforces doubly-stochastic constraints—both dimensions sum to 1—preserving total "mass" across the routing operation. This stabilization proves critical for training deep networks with cross-lane mixing.

### Where does the "hyper-connection" name originate?

The term reflects that these connections operate *above* standard residual pathways, creating higher-order interaction patterns between parallel lanes rather than simple skip connections. The prefix "hyper" indicates the learned, multiplicative gating and structured mixing that distinguishes them from additive residuals.