# Multi-lane Hyper-connections and Sinkhorn-routed Residual Streams in Needle 2: Complete Technical Guide

> Explore Needle 2's multi-lane hyper-connections and Sinkhorn-routed residual streams. Learn how this architecture efficiently models long-range dependencies, avoiding quadratic attention costs.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-22

---

**Needle 2 introduces parallel processing lanes connected by hyper-graph topology and optimal transport-based residual routing to enable efficient long-range dependency modeling without quadratic attention costs.**

The `cactus-compute/needle` repository implements a novel transformer architecture that departs from standard unidirectional attention mechanisms. Needle 2 leverages multi-lane hyper-connections to distribute computation across specialized pathways while employing Sinkhorn-routed residual streams to dynamically re-weight information flow between layers.

## What Are Multi-lane Hyper-connections?

Multi-lane hyper-connections replace the single attention pathway found in traditional transformers with multiple parallel "lanes" that process token sequences simultaneously. Each lane operates as an independent transformer block with distinct routing patterns, allowing the model to specialize different lanes for distinct linguistic features such as syntax, semantics, or positional reasoning.

### The Hyper-connection Graph

Rather than isolating these lanes, Needle 2 implements a hyper-connection graph that enables full connectivity between any pair of lanes at every transformer block. This architecture is implemented in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) within the `MultiLaneTransformer` class. The `HyperMixer` module concatenates lane outputs and projects them through learned linear transformations across the lane dimension, ensuring information exchange without incurring the quadratic cost of standard self-attention across the full sequence length.

## Understanding Sinkhorn-routed Residual Streams

After hyper-connection mixing, Needle 2 does not perform simple residual addition. Instead, the residual signal passes through a `SinkhornRouter` that re-weights the stream using an entropy-regularized optimal transport matrix. The router computes transport costs between the current hidden states and learned routing prototypes, then applies the Sinkhorn algorithm to derive a soft assignment matrix.

### Transport Matrix and Routing Prototypes

The `SinkhornRouter` class, defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), maintains learnable prototypes as parameters (`self.prototypes`) and uses a configurable entropy regularization coefficient (`self.eps`). During the forward pass, the router computes pairwise distances between mixed representations and prototypes using `torch.cdist`, then iteratively normalizes the cost matrix to produce the transport plan. This matrix determines how residual information distributes across the lanes, suppressing noisy pathways while amplifying salient features.

## Architecture Flow: Step-by-Step Integration

The Needle 2 forward pass follows a strict five-stage pipeline implemented across [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) and [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py):

1. **Token Embedding**: Input tokens embed into a shared representation space fed identically into all lanes.
2. **Lane-specific Processing**: Each lane executes independent self-attention (often using FlashAttention optimizations) through its `LaneTransformer` module.
3. **Hyper-connection Aggregation**: The `HyperMixer` concatenates lane outputs along a dedicated dimension and applies cross-lane projections.
4. **Sinkhorn Routing**: The mixed representation multiplies with the Sinkhorn-derived transport matrix from `SinkhornRouter` before residual addition.
5. **Feed-forward Processing**: Layer normalization and position-wise feed-forward networks complete the block.

Repeating these stages across deep stacks yields a residual stream where information continuously flows through dynamically re-weighted pathways.

## Implementation Details and Source Code

The core architectural innovations reside in specific modules within the `cactus-compute/needle` codebase.

### Core Classes in architecture.py

The [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) file defines the primary architectural components:

```python
class MultiLaneTransformer(nn.Module):
    def __init__(self, config):
        self.lanes = nn.ModuleList([LaneTransformer(config) for _ in range(config.num_lanes)])
        self.hyper_mixer = HyperMixer(config)
        self.router = SinkhornRouter(config)

    def forward(self, x):
        lane_outs = [lane(x) for lane in self.lanes]
        mixed = self.hyper_mixer(torch.stack(lane_outs, dim=1))
        routed = self.router(mixed, x)
        return routed

```

This class orchestrates the parallel lane execution and subsequent mixing and routing operations.

### The SinkhornRouter Implementation

The optimal transport mechanism lives in the same file:

```python
class SinkhornRouter(nn.Module):
    def __init__(self, config):
        self.prototypes = nn.Parameter(torch.randn(config.num_prototypes, config.hidden_dim))
        self.eps = 0.1

    def forward(self, mixed, residual):
        cost = torch.cdist(mixed, self.prototypes, p=2)
        transport = sinkhorn(cost, eps=self.eps, n_iters=3)
        routed_residual = torch.einsum('bhk,bh->bhk', transport, residual)
        return mixed + routed_residual

```

The `sinkhorn` function itself resides in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py), providing the iterative matrix scaling operations required for entropy-regularized optimal transport.

## Working with Needle 2: Practical Code Examples

The [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) module exposes high-level APIs that abstract the underlying multi-lane complexity while allowing diagnostic access to internal states.

### Loading a Pre-trained Model

```python
from needle.model.run import NeedleModel

model = NeedleModel.from_pretrained("cactus-compute/needle-v2")
model.eval()

```

### Performing Inference

```python
prompt = "Explain quantum tunneling in simple terms."
tokens = model.tokenizer.encode(prompt, return_tensors="pt")

with torch.no_grad():
    logits = model(tokens)

generated = model.tokenizer.decode(logits.argmax(-1).squeeze())
print(generated)

```

### Inspecting Lane Activations

For research and debugging, enable lane-level diagnostics:

```python
model = NeedleModel.from_pretrained("cactus-compute/needle-v2", return_lane_activations=True)
tokens = model.tokenizer.encode("Example input", return_tensors="pt")

with torch.no_grad():
    outputs = model(tokens)

lane_acts = outputs["lane_activations"]
for i, act in enumerate(lane_acts):
    print(f"Lane {i} mean activation: {act.mean().item():.4f}")

```

### Customizing Lane Count

Modify the configuration before instantiation:

```python
from needle.model.architecture import MultiLaneTransformer, Config

cfg = Config.from_pretrained("cactus-compute/needle-v2")
cfg.num_lanes = 8

model = MultiLaneTransformer(cfg)

```

## Summary

- **Multi-lane hyper-connections** enable parallel transformer processing across specialized pathways, implemented in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) via `MultiLaneTransformer` and `HyperMixer`.
- **Sinkhorn-routed residual streams** replace naive residual addition with optimal transport-based re-weighting using the `SinkhornRouter` class and utilities from [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py).
- The architecture maintains computational efficiency by avoiding quadratic attention costs while increasing model capacity through lane specialization.
- Users interact with these features through the `NeedleModel` class in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), which supports both standard inference and deep introspection of lane activations.

## Frequently Asked Questions

### How do multi-lane hyper-connections reduce computational complexity compared to standard attention?

Multi-lane hyper-connections distribute the sequence processing load across parallel pathways, each handling the full sequence but with reduced per-lane dimensionality. The hyper-connection graph facilitates information exchange between lanes through learned linear projections rather than full pairwise attention matrices, avoiding the quadratic scaling of sequence length that plagues standard self-attention mechanisms.

### What is the purpose of the Sinkhorn algorithm in the residual routing mechanism?

The Sinkhorn algorithm computes an entropy-regularized optimal transport plan that determines how residual information should be distributed across lanes and prototypes. This differentiable routing mechanism allows the model to adaptively suppress uninformative residual pathways while amplifying relevant ones during both forward inference and backpropagation, improving gradient flow stability.

### Where can I find the configuration parameters for adjusting the number of lanes or Sinkhorn iterations?

Configuration parameters reside in the `Config` class accessed via `Config.from_pretrained()` in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py). Key attributes include `num_lanes` for controlling parallelism and `eps` (within `SinkhornRouter`) for regulating transport entropy. The [`quantize.py`](https://github.com/cactus-compute/needle/blob/main/quantize.py) module exposes the `n_iters` parameter controlling Sinkhorn iteration count, which trades off routing precision against computational overhead.

### How does Needle 2 differ from the original Needle architecture?

Needle 2 specifically introduces the multi-lane parallelization strategy and Sinkhorn-based residual routing, whereas the original Needle implementation utilized standard single-path transformer blocks with conventional residual connections. The second major release transitions from monolithic attention to the specialized lane-and-mixer architecture described in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py).